AI Final Course Project - Smoke Detection



Introduction

Binary classification using Tensorflow/Keras is an important part of machine learning used to identify and distinguish between two distinct classes. Smoke detection is a critical problem in public safety, and detecting smoke early can help prevent fires and save lives. Machine learning models can be trained to recognize patterns in images that correspond to smoke, which can be used to classify new images as containing smoke or not. Tensorflow/Keras simplifies building machine learning models, and the problem of smoke detection can be solved using various techniques.

Dataset Source

The dataset used in this AI project was sourced from Kaggle, a popular online community of data scientists and machine learning practitioners.



Loading Python Libraries 

# Let's import the required python packages for next day rain predict
import shap
import random
import warnings
import numpy as np
import pandas as pd
import seaborn as sns
import missingno as msno
import matplotlib.pyplot as plt
from imblearn.over_sampling import SMOTE
from pandas_profiling import ProfileReport
from lime.lime_tabular import LimeTabularExplainer

%matplotlib inline

# Tensorflow/Keras imports
import tensorflow as tf
from tensorflow.keras.layers import Dense
from tensorflow.keras.models import Sequential
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
`import pandas_profiling` is going to be deprecated by April 1st. Please use `import ydata_profiling` instead.



Importing the Data

# Let's load the smoke detection data into a dataframe using pandas
data_frame = pd.read_csv("smoke_detection.csv", index_col=0)
plot_data_frame = data_frame.copy()

# Let's take a quick look at the shape of the dataframe
print("Smoke detection data shape -->", data_frame.shape)
print()

# Let's take a brief look at the contents of the dataframe
data_frame.head(10)
Smoke detection data shape --> (62630, 15)
UTC Temperature[C] Humidity[%] TVOC[ppb] eCO2[ppm] Raw H2 Raw Ethanol Pressure[hPa] PM1.0 PM2.5 NC0.5 NC1.0 NC2.5 CNT Fire Alarm
0 1654733331 20.000 57.36 0 400 12306 18520 939.735 0.0 0.00 0.0 0.000 0.00 0 0
1 1654733332 20.015 56.67 0 400 12345 18651 939.744 0.0 0.00 0.0 0.000 0.00 1 0
2 1654733333 20.029 55.96 0 400 12374 18764 939.738 0.0 0.00 0.0 0.000 0.00 2 0
3 1654733334 20.044 55.28 0 400 12390 18849 939.736 0.0 0.00 0.0 0.000 0.00 3 0
4 1654733335 20.059 54.69 0 400 12403 18921 939.744 0.0 0.00 0.0 0.000 0.00 4 0
5 1654733336 20.073 54.12 0 400 12419 18998 939.725 0.0 0.00 0.0 0.000 0.00 5 0
6 1654733337 20.088 53.61 0 400 12432 19058 939.738 0.0 0.00 0.0 0.000 0.00 6 0
7 1654733338 20.103 53.20 0 400 12439 19114 939.758 0.0 0.00 0.0 0.000 0.00 7 0
8 1654733339 20.117 52.81 0 400 12448 19155 939.758 0.0 0.00 0.0 0.000 0.00 8 0
9 1654733340 20.132 52.46 0 400 12453 19195 939.756 0.9 3.78 0.0 4.369 2.78 9 0



Gaining Insights from Data

# Let's obtain a brief overview of the dataframe
data_frame.info()
<class 'pandas.core.frame.DataFrame'>
Int64Index: 62630 entries, 0 to 62629
Data columns (total 15 columns):
 #   Column          Non-Null Count  Dtype  
---  ------          --------------  -----  
 0   UTC             62630 non-null  int64  
 1   Temperature[C]  62630 non-null  float64
 2   Humidity[%]     62630 non-null  float64
 3   TVOC[ppb]       62630 non-null  int64  
 4   eCO2[ppm]       62630 non-null  int64  
 5   Raw H2          62630 non-null  int64  
 6   Raw Ethanol     62630 non-null  int64  
 7   Pressure[hPa]   62630 non-null  float64
 8   PM1.0           62630 non-null  float64
 9   PM2.5           62630 non-null  float64
 10  NC0.5           62630 non-null  float64
 11  NC1.0           62630 non-null  float64
 12  NC2.5           62630 non-null  float64
 13  CNT             62630 non-null  int64  
 14  Fire Alarm      62630 non-null  int64  
dtypes: float64(8), int64(7)
memory usage: 7.6 MB

# Let's see descriptive statistics for all numeric columns
data_frame.describe()
UTC Temperature[C] Humidity[%] TVOC[ppb] eCO2[ppm] Raw H2 Raw Ethanol Pressure[hPa] PM1.0 PM2.5 NC0.5 NC1.0 NC2.5 CNT Fire Alarm
count 6.263000e+04 62630.000000 62630.000000 62630.000000 62630.000000 62630.000000 62630.000000 62630.000000 62630.000000 62630.000000 62630.000000 62630.000000 62630.000000 62630.000000 62630.000000
mean 1.654792e+09 15.970424 48.539499 1942.057528 670.021044 12942.453936 19754.257912 938.627649 100.594309 184.467770 491.463608 203.586487 80.049042 10511.386157 0.714626
std 1.100025e+05 14.359576 8.865367 7811.589055 1905.885439 272.464305 609.513156 1.331344 922.524245 1976.305615 4265.661251 2214.738556 1083.383189 7597.870997 0.451596
min 1.654712e+09 -22.010000 10.740000 0.000000 400.000000 10668.000000 15317.000000 930.852000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000 0.000000
25% 1.654743e+09 10.994250 47.530000 130.000000 400.000000 12830.000000 19435.000000 938.700000 1.280000 1.340000 8.820000 1.384000 0.033000 3625.250000 0.000000
50% 1.654762e+09 20.130000 50.150000 981.000000 400.000000 12924.000000 19501.000000 938.816000 1.810000 1.880000 12.450000 1.943000 0.044000 9336.000000 1.000000
75% 1.654778e+09 25.409500 53.240000 1189.000000 438.000000 13109.000000 20078.000000 939.418000 2.090000 2.180000 14.420000 2.249000 0.051000 17164.750000 1.000000
max 1.655130e+09 59.930000 75.200000 60000.000000 60000.000000 13803.000000 21410.000000 939.861000 14333.690000 45432.260000 61482.030000 51914.680000 30026.438000 24993.000000 1.000000

Observations

  1. The values in the Temperature column include negative numbers.
  2. Datapoints in TVOC, eCO2, PM2.5, NC0.5 coulmns are more spread out or widely dispersed from the mean (average) of the values.



Data Cleaning

# Let's drop the unnecessary columns like UTC and CNT, because UTC is the 
# time when experiment was performed and CNT is auto incremented value

data_frame.drop(["UTC", "CNT"], axis=1, inplace=True)

Check Duplicate Values

# Let's check for the duplicate values
data_frame[data_frame.duplicated(keep=False)]
Temperature[C] Humidity[%] TVOC[ppb] eCO2[ppm] Raw H2 Raw Ethanol Pressure[hPa] PM1.0 PM2.5 NC0.5 NC1.0 NC2.5 Fire Alarm
51335 26.90 45.15 22 415 12846 20760 937.582 2.04 2.12 14.05 2.190 0.049 0
51385 27.12 44.50 55 412 12832 20717 937.571 2.15 2.23 14.80 2.308 0.052 0
57079 26.90 45.15 22 415 12846 20760 937.582 2.04 2.12 14.05 2.190 0.049 0
57129 27.12 44.50 55 412 12832 20717 937.571 2.15 2.23 14.80 2.308 0.052 0

Drop duplicate values

# Let's drop the duplicate values from the dataframe
data_frame = data_frame.drop_duplicates()
# Let's take a quick look at the shape of the dataframe
print("Smoke detection data shape -->", data_frame.shape)
print()

# Let's take a brief look at the contents of the dataframe
data_frame.head(10)
Smoke detection data shape --> (62628, 13)
Temperature[C] Humidity[%] TVOC[ppb] eCO2[ppm] Raw H2 Raw Ethanol Pressure[hPa] PM1.0 PM2.5 NC0.5 NC1.0 NC2.5 Fire Alarm
0 20.000 57.36 0 400 12306 18520 939.735 0.0 0.00 0.0 0.000 0.00 0
1 20.015 56.67 0 400 12345 18651 939.744 0.0 0.00 0.0 0.000 0.00 0
2 20.029 55.96 0 400 12374 18764 939.738 0.0 0.00 0.0 0.000 0.00 0
3 20.044 55.28 0 400 12390 18849 939.736 0.0 0.00 0.0 0.000 0.00 0
4 20.059 54.69 0 400 12403 18921 939.744 0.0 0.00 0.0 0.000 0.00 0
5 20.073 54.12 0 400 12419 18998 939.725 0.0 0.00 0.0 0.000 0.00 0
6 20.088 53.61 0 400 12432 19058 939.738 0.0 0.00 0.0 0.000 0.00 0
7 20.103 53.20 0 400 12439 19114 939.758 0.0 0.00 0.0 0.000 0.00 0
8 20.117 52.81 0 400 12448 19155 939.758 0.0 0.00 0.0 0.000 0.00 0
9 20.132 52.46 0 400 12453 19195 939.756 0.9 3.78 0.0 4.369 2.78 0 
# Function to get unique_counts based on specific column
def value_counts(column_name):
    return data_frame.loc[
        :, column_name
    ].value_counts()  # Returns the unique value counts
# Iterates over all the columns of the dataframe and calls value_counts func
required_columns = [
    "Temperature[C]",
    "Humidity[%]",
    "eCO2[ppm]",
    "Fire Alarm",
]
for column_name in required_columns:
    print(f"Value Counts of {column_name}", sep="\n")
    print(value_counts(column_name=column_name))
    print()
Value Counts of Temperature[C]
 24.480    222
 24.510    206
 24.470    193
 26.950    191
 26.980    189
          ... 
 24.866      1
 14.089      1
-8.907       1
 14.099      1
 16.957      1
Name: Temperature[C], Length: 21672, dtype: int64

Value Counts of Humidity[%]
47.50    130
47.45    130
47.86    126
53.12    126
47.76    126
        ... 
32.07      1
31.77      1
31.59      1
31.35      1
21.61      1
Name: Humidity[%], Length: 3890, dtype: int64

Value Counts of eCO2[ppm]
400     31922
404       942
401       925
406       918
408       896
        ...  
2231        1
1436        1
1602        1
2501        1
1844        1
Name: eCO2[ppm], Length: 1713, dtype: int64

Value Counts of Fire Alarm
1    44757
0    17871
Name: Fire Alarm, dtype: int64

See how many missing data points we have

# Let's get the number of missing data points per column
data_frame.isnull().sum()
Temperature[C]    0
Humidity[%]       0
TVOC[ppb]         0
eCO2[ppm]         0
Raw H2            0
Raw Ethanol       0
Pressure[hPa]     0
PM1.0             0
PM2.5             0
NC0.5             0
NC1.0             0
NC2.5             0
Fire Alarm        0
dtype: int64



Missing data points visualization

# Let's visualize the missing points using barplot
msno.bar(data_frame)
plt.show()



Feature Engineering

Oversampling

# Let's increase the number of data instances associated with label 0 in the 
# FireAlarm feature by oversampling, relative to those with label 1.

oversample = SMOTE()
X_smote, y_smote = oversample.fit_resample(data_frame.iloc[:, :-1].values, data_frame.iloc[:, -1].values)

# Let's create a dataframe by merging X_smote and y_smote
X_normalized_df = pd.DataFrame(X_smote, columns = data_frame.columns.tolist()[:-1])
y_normalized_df = pd.DataFrame(y_smote, columns=["Fire Alarm"])
normalized_dataframe = pd.concat([X_normalized_df, y_normalized_df], axis = 1)



Count Plot of Fire Alarm Label - Before Sampling

# Let's plot the counts of output labels using the countplot
sns.countplot(data=data_frame, x="Fire Alarm")
plt.show()

Count Plot of Fire Alarm Label - After Sampling

# Let's plot the counts of output labels using the countplot
sns.countplot(x=y_smote)
plt.show()

Output Column Distribution

# Let's see the percentage of 0's and 1's in output column(RainTomorrow)
percentage = data_frame.loc[:, "Fire Alarm"].value_counts(normalize=True) * 100
percentage
1    71.46484
0    28.53516
Name: Fire Alarm, dtype: float64

Pie chart distribution - Before Sampling

plt.pie(percentage, labels=["0", "1"], autopct="%1.1f%%")
plt.title("Distribution of FireAlarm label")
plt.show()

# Let's see the percentage of 0's and 1's in output column(RainTomorrow)
percentage = normalized_dataframe.loc[:, "Fire Alarm"].value_counts(normalize=True) * 100
percentage
0    50.0
1    50.0
Name: Fire Alarm, dtype: float64

Pie chart distribution - After Sampling

plt.pie(percentage, labels=["0", "1"], autopct="%1.1f%%")
plt.title("Distribution of FireAlarm label")
plt.show()



Data Normalization

# Function to normalize the values of each column by subtracting
# the mean and dividing by the standard deviation

def z_score_normalization(column_name):
    series = data_frame.loc[:, column_name]
    return (series - series.mean()) / series.std()
# Iterates over all the continuous columns and applies z_score_normalization to each column
for column_name in data_frame.columns:
    if column_name != "Fire Alarm":
        data_frame[column_name] = z_score_normalization(column_name=column_name)

Descriptive statisctics after normalization

# Let's see descriptive statistics for all numeric columns after normalization
data_frame.head()
Temperature[C] Humidity[%] TVOC[ppb] eCO2[ppm] Raw H2 Raw Ethanol Pressure[hPa] PM1.0 PM2.5 NC0.5 NC1.0 NC2.5 Fire Alarm
0 0.280642 0.994913 -0.248616 -0.14168 -2.335897 -2.024990 0.831724 -0.109044 -0.093341 -0.115216 -0.091925 -0.073889 0
1 0.281687 0.917083 -0.248616 -0.14168 -2.192761 -1.810059 0.838484 -0.109044 -0.093341 -0.115216 -0.091925 -0.073889 0
2 0.282662 0.836997 -0.248616 -0.14168 -2.086326 -1.624660 0.833977 -0.109044 -0.093341 -0.115216 -0.091925 -0.073889 0
3 0.283706 0.760295 -0.248616 -0.14168 -2.027604 -1.485201 0.832475 -0.109044 -0.093341 -0.115216 -0.091925 -0.073889 0
4 0.284751 0.693745 -0.248616 -0.14168 -1.979892 -1.367071 0.838484 -0.109044 -0.093341 -0.115216 -0.091925 -0.073889 0



Data Visualizations(Distributions)

Histogram plots before normalization

# Let's see the distributions of all the features in a dataframe using histogram plots
plot_data_frame.hist(bins=10, figsize=(18, 12))
plt.show()

Box plots before normalization

# Let's confirm that all values in a dataframe are within the range of 0 to 1 using boxplots
row = 0
column = 0
nrow_plots = 5
ncol_plots = 3
figure, axis = plt.subplots(nrow_plots, ncol_plots, figsize=(20, 30))
plt.subplots_adjust(wspace=0.5)
for column_name in data_frame.columns:
    sns.boxplot(data=plot_data_frame[column_name], ax=axis[row][column], color="green")
    if column < ncol_plots - 1:
        column += 1
        axis[row][column].set_xlabel(column_name)
    else:
        row += 1
        column = 0

# # Let's remove empty subplot
figure.delaxes(axis[nrow_plots - 1][ncol_plots - 1])
figure.delaxes(axis[nrow_plots - 1][ncol_plots - 2])
figure.delaxes(axis[nrow_plots - 1][ncol_plots - 3])

Histogram plots after normalization

# Let's confirm that all values in a dataframe are within the range of 0 to 1 using histplots
# plt.figure(figsize=(20, 12))
data_frame.hist(bins=10, figsize=(18, 12))
plt.show()

Box plots after normalization

# Let's confirm that all values in a dataframe are within the range of 0 to 1 using boxplots
row = 0
column = 0
nrow_plots = 5
ncol_plots = 3
figure, axis = plt.subplots(nrow_plots, ncol_plots, figsize=(20, 30))
plt.subplots_adjust(wspace=0.5)
for column_name in data_frame.columns:
    sns.boxplot(data=data_frame[column_name], ax=axis[row][column], color="green")
    if column < ncol_plots - 1:
        column += 1
        axis[row][column].set_xlabel(column_name)
    else:
        row += 1
        column = 0

# # Let's remove empty subplot
figure.delaxes(axis[nrow_plots - 1][ncol_plots - 1])
figure.delaxes(axis[nrow_plots - 1][ncol_plots - 2])
figure.delaxes(axis[nrow_plots - 1][ncol_plots - 3])

Pandas Profiling

# Let's generate and display the pandas_profile of the entire rain dataframe
profile = ProfileReport(data_frame)
profile
{"ascii":false,"bar_format":null,"colour":null,"elapsed":7.293701171875e-3,"initial":0,"n":0,"ncols":null,"nrows":null,"postfix":null,"prefix":"Summarize dataset","rate":null,"total":5,"unit":"it","unit_divisor":1000,"unit_scale":false}
{"ascii":false,"bar_format":null,"colour":null,"elapsed":8.024930953979492e-3,"initial":0,"n":0,"ncols":null,"nrows":null,"postfix":null,"prefix":"Generate report structure","rate":null,"total":1,"unit":"it","unit_divisor":1000,"unit_scale":false}
{"ascii":false,"bar_format":null,"colour":null,"elapsed":6.735801696777344e-3,"initial":0,"n":0,"ncols":null,"nrows":null,"postfix":null,"prefix":"Render HTML","rate":null,"total":1,"unit":"it","unit_divisor":1000,"unit_scale":false}



Phase - 2 Build a model to overfit the entire dataset)

# Let's define a function to plot the loss and accuracy
def plot_loss_accuracy(model_name, model, model_history):
    figure, (ax1, ax2) = plt.subplots(1, 2, figsize=(18, 6))
    figure.suptitle(model_name)
    ax1.plot(model_history.history["loss"])
    ax1.set_ylabel("loss")
    ax1.set_xlabel("epoch")
    ax1.legend(["Overall loss data"], loc="best")

    ax2.plot(model_history.history["accuracy"])
    ax2.set_ylabel("accuracy")
    ax2.set_xlabel("epoch")
    ax2.legend(["Overall accuracy data"], loc="best")
    figure.show()

Model Building with one layer and one neuron (1-1) and 20 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layer to the model network
model.add(Dense(1, input_dim=data_frame.shape[1] - 1, activation="sigmoid"))
2023-05-09 00:04:41.066101: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:267] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected

# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process

model.compile(loss="binary_crossentropy", optimizer="rmsprop", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=20, verbose=1)
Epoch 1/20
2798/2798 [==============================] - 3s 821us/step - loss: 219.0697 - accuracy: 0.7485
Epoch 2/20
2798/2798 [==============================] - 2s 821us/step - loss: 26.6872 - accuracy: 0.7787
Epoch 3/20
2798/2798 [==============================] - 2s 814us/step - loss: 14.7462 - accuracy: 0.7829
Epoch 4/20
2798/2798 [==============================] - 2s 818us/step - loss: 7.1313 - accuracy: 0.7913
Epoch 5/20
2798/2798 [==============================] - 2s 826us/step - loss: 5.5498 - accuracy: 0.7926
Epoch 6/20
2798/2798 [==============================] - 2s 837us/step - loss: 5.4974 - accuracy: 0.7956
Epoch 7/20
2798/2798 [==============================] - 2s 843us/step - loss: 5.1337 - accuracy: 0.7966
Epoch 8/20
2798/2798 [==============================] - 2s 820us/step - loss: 5.1542 - accuracy: 0.7995
Epoch 9/20
2798/2798 [==============================] - 2s 845us/step - loss: 5.1432 - accuracy: 0.8026
Epoch 10/20
2798/2798 [==============================] - 2s 823us/step - loss: 4.9250 - accuracy: 0.8043
Epoch 11/20
2798/2798 [==============================] - 2s 811us/step - loss: 4.8836 - accuracy: 0.8048
Epoch 12/20
2798/2798 [==============================] - 2s 821us/step - loss: 4.8007 - accuracy: 0.8077
Epoch 13/20
2798/2798 [==============================] - 2s 820us/step - loss: 4.7748 - accuracy: 0.8087
Epoch 14/20
2798/2798 [==============================] - 2s 820us/step - loss: 4.7512 - accuracy: 0.8071
Epoch 15/20
2798/2798 [==============================] - 2s 819us/step - loss: 4.6464 - accuracy: 0.8128
Epoch 16/20
2798/2798 [==============================] - 2s 821us/step - loss: 4.6174 - accuracy: 0.8135
Epoch 17/20
2798/2798 [==============================] - 2s 816us/step - loss: 4.5481 - accuracy: 0.8122
Epoch 18/20
2798/2798 [==============================] - 2s 822us/step - loss: 4.6049 - accuracy: 0.8173
Epoch 19/20
2798/2798 [==============================] - 2s 818us/step - loss: 4.5058 - accuracy: 0.8142
Epoch 20/20
2798/2798 [==============================] - 2s 828us/step - loss: 4.4197 - accuracy: 0.8166

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_base", model=model, model_history=history)

Observations

If the training accuracy of a neural network is improving gradually, one option to potentially improve it further is to increase the number of epochs or increase the complexity of the model architecture by adding more layers and neurons



Model Building with one layer and one neuron (1-1) and 50 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layer to the model network
model.add(Dense(1, input_dim=data_frame.shape[1] - 1, activation="sigmoid"))
# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process

model.compile(loss="binary_crossentropy", optimizer="rmsprop", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=50, verbose=1)
Epoch 1/50
2798/2798 [==============================] - 3s 822us/step - loss: 647.4402 - accuracy: 0.5564
Epoch 2/50
2798/2798 [==============================] - 2s 821us/step - loss: 9.7232 - accuracy: 0.8102
Epoch 3/50
2798/2798 [==============================] - 2s 821us/step - loss: 3.7189 - accuracy: 0.8175
Epoch 4/50
2798/2798 [==============================] - 2s 848us/step - loss: 3.5089 - accuracy: 0.8195
Epoch 5/50
2798/2798 [==============================] - 2s 863us/step - loss: 3.5129 - accuracy: 0.8195
Epoch 6/50
2798/2798 [==============================] - 2s 820us/step - loss: 3.5580 - accuracy: 0.8176
Epoch 7/50
2798/2798 [==============================] - 2s 825us/step - loss: 3.5295 - accuracy: 0.8198
Epoch 8/50
2798/2798 [==============================] - 2s 823us/step - loss: 3.5048 - accuracy: 0.8189
Epoch 9/50
2798/2798 [==============================] - 2s 825us/step - loss: 3.5021 - accuracy: 0.8177
Epoch 10/50
2798/2798 [==============================] - 2s 820us/step - loss: 3.5531 - accuracy: 0.8167
Epoch 11/50
2798/2798 [==============================] - 2s 821us/step - loss: 3.4740 - accuracy: 0.8165
Epoch 12/50
2798/2798 [==============================] - 2s 828us/step - loss: 3.5100 - accuracy: 0.8170
Epoch 13/50
2798/2798 [==============================] - 2s 832us/step - loss: 3.4642 - accuracy: 0.8158
Epoch 14/50
2798/2798 [==============================] - 2s 829us/step - loss: 3.5107 - accuracy: 0.8174
Epoch 15/50
2798/2798 [==============================] - 2s 828us/step - loss: 3.5664 - accuracy: 0.8158
Epoch 16/50
2798/2798 [==============================] - 2s 820us/step - loss: 3.4947 - accuracy: 0.8149
Epoch 17/50
2798/2798 [==============================] - 2s 827us/step - loss: 3.5002 - accuracy: 0.8139
Epoch 18/50
2798/2798 [==============================] - 2s 830us/step - loss: 3.5377 - accuracy: 0.8121
Epoch 19/50
2798/2798 [==============================] - 2s 828us/step - loss: 3.5013 - accuracy: 0.8144
Epoch 20/50
2798/2798 [==============================] - 2s 832us/step - loss: 3.4693 - accuracy: 0.8151
Epoch 21/50
2798/2798 [==============================] - 2s 837us/step - loss: 3.4781 - accuracy: 0.8155
Epoch 22/50
2798/2798 [==============================] - 2s 835us/step - loss: 3.4813 - accuracy: 0.8154
Epoch 23/50
2798/2798 [==============================] - 2s 834us/step - loss: 3.4699 - accuracy: 0.8125
Epoch 24/50
2798/2798 [==============================] - 2s 824us/step - loss: 3.5329 - accuracy: 0.8111
Epoch 25/50
2798/2798 [==============================] - 2s 829us/step - loss: 3.4905 - accuracy: 0.8114
Epoch 26/50
2798/2798 [==============================] - 2s 823us/step - loss: 3.4687 - accuracy: 0.8128
Epoch 27/50
2798/2798 [==============================] - 2s 827us/step - loss: 3.4625 - accuracy: 0.8131
Epoch 28/50
2798/2798 [==============================] - 2s 835us/step - loss: 3.4602 - accuracy: 0.8142
Epoch 29/50
2798/2798 [==============================] - 2s 837us/step - loss: 3.4646 - accuracy: 0.8120
Epoch 30/50
2798/2798 [==============================] - 2s 823us/step - loss: 3.4640 - accuracy: 0.8116
Epoch 31/50
2798/2798 [==============================] - 2s 826us/step - loss: 3.4574 - accuracy: 0.8140
Epoch 32/50
2798/2798 [==============================] - 2s 827us/step - loss: 3.5797 - accuracy: 0.8103
Epoch 33/50
2798/2798 [==============================] - 2s 825us/step - loss: 3.5023 - accuracy: 0.8107
Epoch 34/50
2798/2798 [==============================] - 2s 821us/step - loss: 3.4558 - accuracy: 0.8124
Epoch 35/50
2798/2798 [==============================] - 2s 830us/step - loss: 3.5386 - accuracy: 0.8095
Epoch 36/50
2798/2798 [==============================] - 2s 823us/step - loss: 3.5129 - accuracy: 0.8131
Epoch 37/50
2798/2798 [==============================] - 2s 824us/step - loss: 3.4981 - accuracy: 0.8098
Epoch 38/50
2798/2798 [==============================] - 2s 826us/step - loss: 3.5047 - accuracy: 0.8094
Epoch 39/50
2798/2798 [==============================] - 2s 828us/step - loss: 3.4877 - accuracy: 0.8141
Epoch 40/50
2798/2798 [==============================] - 2s 821us/step - loss: 3.4948 - accuracy: 0.8093
Epoch 41/50
2798/2798 [==============================] - 2s 827us/step - loss: 3.4619 - accuracy: 0.8116
Epoch 42/50
2798/2798 [==============================] - 2s 834us/step - loss: 3.5001 - accuracy: 0.8078
Epoch 43/50
2798/2798 [==============================] - 2s 827us/step - loss: 3.4762 - accuracy: 0.8109
Epoch 44/50
2798/2798 [==============================] - 2s 822us/step - loss: 3.4903 - accuracy: 0.8100
Epoch 45/50
2798/2798 [==============================] - 2s 826us/step - loss: 3.4968 - accuracy: 0.8096
Epoch 46/50
2798/2798 [==============================] - 2s 827us/step - loss: 3.4817 - accuracy: 0.8101
Epoch 47/50
2798/2798 [==============================] - 2s 837us/step - loss: 3.4727 - accuracy: 0.8101
Epoch 48/50
2798/2798 [==============================] - 2s 826us/step - loss: 3.4710 - accuracy: 0.8121
Epoch 49/50
2798/2798 [==============================] - 2s 834us/step - loss: 3.4335 - accuracy: 0.8094
Epoch 50/50
2798/2798 [==============================] - 2s 822us/step - loss: 3.4504 - accuracy: 0.8090

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_base_50", model=model, model_history=history)

Observations

If the training accuracy of a neural network is improving gradually, one option to potentially improve it further is to increase the number of epochs or increase the complexity of the model architecture by adding more layers and neurons



Model Building with one layer and one neuron (1-1) and 100 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layer to the model network
model.add(Dense(1, input_dim=data_frame.shape[1] - 1, activation="sigmoid"))
# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process

model.compile(loss="binary_crossentropy", optimizer="rmsprop", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=100, verbose=1)
Epoch 1/100
2798/2798 [==============================] - 3s 832us/step - loss: 403.4251 - accuracy: 0.6656
Epoch 2/100
2798/2798 [==============================] - 2s 818us/step - loss: 23.6685 - accuracy: 0.7387
Epoch 3/100
2798/2798 [==============================] - 2s 820us/step - loss: 17.1656 - accuracy: 0.7481
Epoch 4/100
2798/2798 [==============================] - 2s 818us/step - loss: 11.0080 - accuracy: 0.7486
Epoch 5/100
2798/2798 [==============================] - 2s 824us/step - loss: 7.0911 - accuracy: 0.7395
Epoch 6/100
2798/2798 [==============================] - 2s 835us/step - loss: 6.3712 - accuracy: 0.7413
Epoch 7/100
2798/2798 [==============================] - 2s 825us/step - loss: 6.2918 - accuracy: 0.7424
Epoch 8/100
2798/2798 [==============================] - 2s 823us/step - loss: 6.1156 - accuracy: 0.7457
Epoch 9/100
2798/2798 [==============================] - 2s 827us/step - loss: 6.0251 - accuracy: 0.7474
Epoch 10/100
2798/2798 [==============================] - 2s 822us/step - loss: 5.8637 - accuracy: 0.7503
Epoch 11/100
2798/2798 [==============================] - 2s 828us/step - loss: 5.8621 - accuracy: 0.7528
Epoch 12/100
2798/2798 [==============================] - 2s 828us/step - loss: 5.9096 - accuracy: 0.7577
Epoch 13/100
2798/2798 [==============================] - 2s 827us/step - loss: 5.7410 - accuracy: 0.7574
Epoch 14/100
2798/2798 [==============================] - 2s 828us/step - loss: 5.6254 - accuracy: 0.7574
Epoch 15/100
2798/2798 [==============================] - 2s 826us/step - loss: 5.5302 - accuracy: 0.7588
Epoch 16/100
2798/2798 [==============================] - 2s 829us/step - loss: 5.4850 - accuracy: 0.7625
Epoch 17/100
2798/2798 [==============================] - 2s 833us/step - loss: 5.6282 - accuracy: 0.7627
Epoch 18/100
2798/2798 [==============================] - 2s 831us/step - loss: 5.3937 - accuracy: 0.7632
Epoch 19/100
2798/2798 [==============================] - 2s 829us/step - loss: 5.3444 - accuracy: 0.7673
Epoch 20/100
2798/2798 [==============================] - 2s 845us/step - loss: 5.4144 - accuracy: 0.7669
Epoch 21/100
2798/2798 [==============================] - 2s 827us/step - loss: 5.2899 - accuracy: 0.7688
Epoch 22/100
2798/2798 [==============================] - 2s 830us/step - loss: 5.2357 - accuracy: 0.7682
Epoch 23/100
2798/2798 [==============================] - 2s 832us/step - loss: 5.2331 - accuracy: 0.7709
Epoch 24/100
2798/2798 [==============================] - 2s 834us/step - loss: 5.1800 - accuracy: 0.7698
Epoch 25/100
2798/2798 [==============================] - 2s 822us/step - loss: 5.0697 - accuracy: 0.7713
Epoch 26/100
2798/2798 [==============================] - 2s 822us/step - loss: 5.1965 - accuracy: 0.7709
Epoch 27/100
2798/2798 [==============================] - 2s 844us/step - loss: 5.1052 - accuracy: 0.7738
Epoch 28/100
2798/2798 [==============================] - 2s 855us/step - loss: 5.0324 - accuracy: 0.7731
Epoch 29/100
2798/2798 [==============================] - 2s 845us/step - loss: 4.9979 - accuracy: 0.7732
Epoch 30/100
2798/2798 [==============================] - 2s 831us/step - loss: 5.0015 - accuracy: 0.7723
Epoch 31/100
2798/2798 [==============================] - 2s 827us/step - loss: 4.9167 - accuracy: 0.7756
Epoch 32/100
2798/2798 [==============================] - 2s 830us/step - loss: 4.9035 - accuracy: 0.7743
Epoch 33/100
2798/2798 [==============================] - 2s 826us/step - loss: 4.9138 - accuracy: 0.7710
Epoch 34/100
2798/2798 [==============================] - 2s 832us/step - loss: 4.8487 - accuracy: 0.7764
Epoch 35/100
2798/2798 [==============================] - 2s 834us/step - loss: 4.8427 - accuracy: 0.7756
Epoch 36/100
2798/2798 [==============================] - 2s 832us/step - loss: 4.7487 - accuracy: 0.7769
Epoch 37/100
2798/2798 [==============================] - 2s 839us/step - loss: 4.7644 - accuracy: 0.7757
Epoch 38/100
2798/2798 [==============================] - 2s 836us/step - loss: 4.7277 - accuracy: 0.7776
Epoch 39/100
2798/2798 [==============================] - 2s 822us/step - loss: 4.6897 - accuracy: 0.7793
Epoch 40/100
2798/2798 [==============================] - 2s 826us/step - loss: 4.6277 - accuracy: 0.7767
Epoch 41/100
2798/2798 [==============================] - 2s 830us/step - loss: 4.6209 - accuracy: 0.7794
Epoch 42/100
2798/2798 [==============================] - 2s 827us/step - loss: 4.6255 - accuracy: 0.7794
Epoch 43/100
2798/2798 [==============================] - 2s 830us/step - loss: 4.6217 - accuracy: 0.7810
Epoch 44/100
2798/2798 [==============================] - 2s 826us/step - loss: 4.5345 - accuracy: 0.7806
Epoch 45/100
2798/2798 [==============================] - 2s 835us/step - loss: 4.5652 - accuracy: 0.7818
Epoch 46/100
2798/2798 [==============================] - 2s 817us/step - loss: 4.4464 - accuracy: 0.7814
Epoch 47/100
2798/2798 [==============================] - 2s 835us/step - loss: 4.4759 - accuracy: 0.7798
Epoch 48/100
2798/2798 [==============================] - 2s 830us/step - loss: 4.4623 - accuracy: 0.7797
Epoch 49/100
2798/2798 [==============================] - 2s 827us/step - loss: 4.4243 - accuracy: 0.7834
Epoch 50/100
2798/2798 [==============================] - 2s 827us/step - loss: 4.4929 - accuracy: 0.7822
Epoch 51/100
2798/2798 [==============================] - 2s 820us/step - loss: 4.3893 - accuracy: 0.7848
Epoch 52/100
2798/2798 [==============================] - 2s 827us/step - loss: 4.4260 - accuracy: 0.7842
Epoch 53/100
2798/2798 [==============================] - 2s 821us/step - loss: 4.2805 - accuracy: 0.7829
Epoch 54/100
2798/2798 [==============================] - 2s 827us/step - loss: 4.3870 - accuracy: 0.7848
Epoch 55/100
2798/2798 [==============================] - 2s 824us/step - loss: 4.2986 - accuracy: 0.7847
Epoch 56/100
2798/2798 [==============================] - 2s 826us/step - loss: 4.3012 - accuracy: 0.7846
Epoch 57/100
2798/2798 [==============================] - 2s 830us/step - loss: 4.2750 - accuracy: 0.7833
Epoch 58/100
2798/2798 [==============================] - 2s 830us/step - loss: 4.3123 - accuracy: 0.7841
Epoch 59/100
2798/2798 [==============================] - 2s 824us/step - loss: 4.2785 - accuracy: 0.7839
Epoch 60/100
2798/2798 [==============================] - 2s 829us/step - loss: 4.1434 - accuracy: 0.7868
Epoch 61/100
2798/2798 [==============================] - 2s 829us/step - loss: 4.1923 - accuracy: 0.7874
Epoch 62/100
2798/2798 [==============================] - 2s 825us/step - loss: 4.1740 - accuracy: 0.7864
Epoch 63/100
2798/2798 [==============================] - 2s 828us/step - loss: 4.1493 - accuracy: 0.7858
Epoch 64/100
2798/2798 [==============================] - 2s 826us/step - loss: 4.1628 - accuracy: 0.7878
Epoch 65/100
2798/2798 [==============================] - 2s 828us/step - loss: 4.1416 - accuracy: 0.7867
Epoch 66/100
2798/2798 [==============================] - 2s 825us/step - loss: 4.0791 - accuracy: 0.7890
Epoch 67/100
2798/2798 [==============================] - 2s 828us/step - loss: 4.0215 - accuracy: 0.7897
Epoch 68/100
2798/2798 [==============================] - 2s 829us/step - loss: 4.1124 - accuracy: 0.7868
Epoch 69/100
2798/2798 [==============================] - 2s 829us/step - loss: 4.0858 - accuracy: 0.7875
Epoch 70/100
2798/2798 [==============================] - 2s 832us/step - loss: 4.0544 - accuracy: 0.7899
Epoch 71/100
2798/2798 [==============================] - 2s 826us/step - loss: 4.0288 - accuracy: 0.7894
Epoch 72/100
2798/2798 [==============================] - 2s 825us/step - loss: 4.0511 - accuracy: 0.7886
Epoch 73/100
2798/2798 [==============================] - 2s 832us/step - loss: 4.0386 - accuracy: 0.7894
Epoch 74/100
2798/2798 [==============================] - 2s 829us/step - loss: 4.0124 - accuracy: 0.7899
Epoch 75/100
2798/2798 [==============================] - 2s 824us/step - loss: 4.0408 - accuracy: 0.7898
Epoch 76/100
2798/2798 [==============================] - 2s 827us/step - loss: 3.9452 - accuracy: 0.7891
Epoch 77/100
2798/2798 [==============================] - 2s 829us/step - loss: 3.9389 - accuracy: 0.7937
Epoch 78/100
2798/2798 [==============================] - 2s 832us/step - loss: 4.0068 - accuracy: 0.7918
Epoch 79/100
2798/2798 [==============================] - 2s 827us/step - loss: 3.9694 - accuracy: 0.7909
Epoch 80/100
2798/2798 [==============================] - 2s 815us/step - loss: 3.9854 - accuracy: 0.7915
Epoch 81/100
2798/2798 [==============================] - 2s 831us/step - loss: 3.8976 - accuracy: 0.7908
Epoch 82/100
2798/2798 [==============================] - 2s 825us/step - loss: 3.9159 - accuracy: 0.7918
Epoch 83/100
2798/2798 [==============================] - 2s 818us/step - loss: 3.9864 - accuracy: 0.7942
Epoch 84/100
2798/2798 [==============================] - 2s 822us/step - loss: 3.9057 - accuracy: 0.7923
Epoch 85/100
2798/2798 [==============================] - 2s 825us/step - loss: 3.9378 - accuracy: 0.7914
Epoch 86/100
2798/2798 [==============================] - 2s 826us/step - loss: 3.8733 - accuracy: 0.7919
Epoch 87/100
2798/2798 [==============================] - 2s 826us/step - loss: 3.8684 - accuracy: 0.7956
Epoch 88/100
2798/2798 [==============================] - 2s 823us/step - loss: 3.8616 - accuracy: 0.7952
Epoch 89/100
2798/2798 [==============================] - 2s 826us/step - loss: 3.8711 - accuracy: 0.7931
Epoch 90/100
2798/2798 [==============================] - 2s 824us/step - loss: 3.8195 - accuracy: 0.7937
Epoch 91/100
2798/2798 [==============================] - 2s 826us/step - loss: 3.8703 - accuracy: 0.7943
Epoch 92/100
2798/2798 [==============================] - 2s 822us/step - loss: 3.8637 - accuracy: 0.7906
Epoch 93/100
2798/2798 [==============================] - 2s 829us/step - loss: 3.8439 - accuracy: 0.7955
Epoch 94/100
2798/2798 [==============================] - 2s 828us/step - loss: 3.8642 - accuracy: 0.7939
Epoch 95/100
2798/2798 [==============================] - 2s 840us/step - loss: 3.8112 - accuracy: 0.7939
Epoch 96/100
2798/2798 [==============================] - 2s 830us/step - loss: 3.8411 - accuracy: 0.7959
Epoch 97/100
2798/2798 [==============================] - 2s 831us/step - loss: 3.8125 - accuracy: 0.7929
Epoch 98/100
2798/2798 [==============================] - 2s 827us/step - loss: 3.7336 - accuracy: 0.7968
Epoch 99/100
2798/2798 [==============================] - 2s 830us/step - loss: 3.7812 - accuracy: 0.7956
Epoch 100/100
2798/2798 [==============================] - 2s 837us/step - loss: 3.7983 - accuracy: 0.7945

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_base_100", model=model, model_history=history)

Observations

The neural network's training accuracy is gradually improving, one possible way to enhance it further is by increasing the number of training epochs.



Model Building with one layer and one neuron (1-1) and 200 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layer to the model network
model.add(Dense(1, input_dim=data_frame.shape[1] - 1, activation="sigmoid"))
# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process

model.compile(loss="binary_crossentropy", optimizer="rmsprop", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=200, verbose=1)
Epoch 1/200
2798/2798 [==============================] - 3s 826us/step - loss: 69.2802 - accuracy: 0.2901
Epoch 2/200
2798/2798 [==============================] - 2s 827us/step - loss: 12.1746 - accuracy: 0.4618
Epoch 3/200
2798/2798 [==============================] - 2s 828us/step - loss: 7.0137 - accuracy: 0.5854
Epoch 4/200
2798/2798 [==============================] - 2s 832us/step - loss: 5.1221 - accuracy: 0.6685
Epoch 5/200
2798/2798 [==============================] - 2s 855us/step - loss: 4.6705 - accuracy: 0.7090
Epoch 6/200
2798/2798 [==============================] - 2s 858us/step - loss: 4.5758 - accuracy: 0.7284
Epoch 7/200
2798/2798 [==============================] - 2s 824us/step - loss: 4.4657 - accuracy: 0.7382
Epoch 8/200
2798/2798 [==============================] - 2s 824us/step - loss: 4.4499 - accuracy: 0.7464
Epoch 9/200
2798/2798 [==============================] - 2s 825us/step - loss: 4.3882 - accuracy: 0.7528
Epoch 10/200
2798/2798 [==============================] - 2s 826us/step - loss: 4.4053 - accuracy: 0.7549
Epoch 11/200
2798/2798 [==============================] - 2s 833us/step - loss: 4.3524 - accuracy: 0.7565
Epoch 12/200
2798/2798 [==============================] - 2s 825us/step - loss: 4.3081 - accuracy: 0.7579
Epoch 13/200
2798/2798 [==============================] - 2s 835us/step - loss: 4.3427 - accuracy: 0.7590
Epoch 14/200
2798/2798 [==============================] - 2s 832us/step - loss: 4.2486 - accuracy: 0.7629
Epoch 15/200
2798/2798 [==============================] - 2s 828us/step - loss: 4.2559 - accuracy: 0.7633
Epoch 16/200
2798/2798 [==============================] - 2s 839us/step - loss: 4.2639 - accuracy: 0.7666
Epoch 17/200
2798/2798 [==============================] - 2s 826us/step - loss: 4.2299 - accuracy: 0.7651
Epoch 18/200
2798/2798 [==============================] - 2s 821us/step - loss: 4.1749 - accuracy: 0.7687
Epoch 19/200
2798/2798 [==============================] - 2s 824us/step - loss: 4.1964 - accuracy: 0.7671
Epoch 20/200
2798/2798 [==============================] - 2s 824us/step - loss: 4.1538 - accuracy: 0.7692
Epoch 21/200
2798/2798 [==============================] - 2s 819us/step - loss: 4.1893 - accuracy: 0.7709
Epoch 22/200
2798/2798 [==============================] - 2s 822us/step - loss: 4.1408 - accuracy: 0.7708
Epoch 23/200
2798/2798 [==============================] - 2s 824us/step - loss: 4.1382 - accuracy: 0.7689
Epoch 24/200
2798/2798 [==============================] - 2s 822us/step - loss: 4.1234 - accuracy: 0.7717
Epoch 25/200
2798/2798 [==============================] - 2s 823us/step - loss: 4.0764 - accuracy: 0.7748
Epoch 26/200
2798/2798 [==============================] - 2s 823us/step - loss: 4.0817 - accuracy: 0.7741
Epoch 27/200
2798/2798 [==============================] - 2s 830us/step - loss: 4.0999 - accuracy: 0.7741
Epoch 28/200
2798/2798 [==============================] - 2s 820us/step - loss: 3.9852 - accuracy: 0.7758
Epoch 29/200
2798/2798 [==============================] - 2s 822us/step - loss: 4.0810 - accuracy: 0.7765
Epoch 30/200
2798/2798 [==============================] - 2s 817us/step - loss: 4.0576 - accuracy: 0.7766
Epoch 31/200
2798/2798 [==============================] - 2s 819us/step - loss: 4.0315 - accuracy: 0.7759
Epoch 32/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.9803 - accuracy: 0.7777
Epoch 33/200
2798/2798 [==============================] - 2s 826us/step - loss: 3.9899 - accuracy: 0.7789
Epoch 34/200
2798/2798 [==============================] - 2s 819us/step - loss: 3.9946 - accuracy: 0.7778
Epoch 35/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.9257 - accuracy: 0.7830
Epoch 36/200
2798/2798 [==============================] - 2s 816us/step - loss: 3.9280 - accuracy: 0.7847
Epoch 37/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.9656 - accuracy: 0.7794
Epoch 38/200
2798/2798 [==============================] - 2s 820us/step - loss: 3.9199 - accuracy: 0.7800
Epoch 39/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.8907 - accuracy: 0.7838
Epoch 40/200
2798/2798 [==============================] - 2s 822us/step - loss: 3.8830 - accuracy: 0.7821
Epoch 41/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.8781 - accuracy: 0.7858
Epoch 42/200
2798/2798 [==============================] - 2s 829us/step - loss: 3.8181 - accuracy: 0.7871
Epoch 43/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.8898 - accuracy: 0.7871
Epoch 44/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.8427 - accuracy: 0.7870
Epoch 45/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.8255 - accuracy: 0.7846
Epoch 46/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.8609 - accuracy: 0.7873
Epoch 47/200
2798/2798 [==============================] - 2s 829us/step - loss: 3.8368 - accuracy: 0.7857
Epoch 48/200
2798/2798 [==============================] - 2s 826us/step - loss: 3.8441 - accuracy: 0.7875
Epoch 49/200
2798/2798 [==============================] - 2s 818us/step - loss: 3.8633 - accuracy: 0.7856
Epoch 50/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.8716 - accuracy: 0.7882
Epoch 51/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.8007 - accuracy: 0.7893
Epoch 52/200
2798/2798 [==============================] - 2s 828us/step - loss: 3.8423 - accuracy: 0.7864
Epoch 53/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.8111 - accuracy: 0.7884
Epoch 54/200
2798/2798 [==============================] - 2s 819us/step - loss: 3.7955 - accuracy: 0.7894
Epoch 55/200
2798/2798 [==============================] - 2s 820us/step - loss: 3.7707 - accuracy: 0.7897
Epoch 56/200
2798/2798 [==============================] - 2s 839us/step - loss: 3.8280 - accuracy: 0.7892
Epoch 57/200
2798/2798 [==============================] - 2s 825us/step - loss: 3.8221 - accuracy: 0.7914
Epoch 58/200
2798/2798 [==============================] - 2s 828us/step - loss: 3.7667 - accuracy: 0.7915
Epoch 59/200
2798/2798 [==============================] - 2s 829us/step - loss: 3.7245 - accuracy: 0.7936
Epoch 60/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.7201 - accuracy: 0.7912
Epoch 61/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.7430 - accuracy: 0.7916
Epoch 62/200
2798/2798 [==============================] - 2s 829us/step - loss: 3.7472 - accuracy: 0.7917
Epoch 63/200
2798/2798 [==============================] - 2s 826us/step - loss: 3.7652 - accuracy: 0.7910
Epoch 64/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.7599 - accuracy: 0.7910
Epoch 65/200
2798/2798 [==============================] - 2s 830us/step - loss: 3.7373 - accuracy: 0.7939
Epoch 66/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.7180 - accuracy: 0.7936
Epoch 67/200
2798/2798 [==============================] - 2s 820us/step - loss: 3.7391 - accuracy: 0.7894
Epoch 68/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.7468 - accuracy: 0.7922
Epoch 69/200
2798/2798 [==============================] - 2s 822us/step - loss: 3.6669 - accuracy: 0.7948
Epoch 70/200
2798/2798 [==============================] - 2s 820us/step - loss: 3.7085 - accuracy: 0.7931
Epoch 71/200
2798/2798 [==============================] - 2s 826us/step - loss: 3.6910 - accuracy: 0.7934
Epoch 72/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.7412 - accuracy: 0.7932
Epoch 73/200
2798/2798 [==============================] - 2s 826us/step - loss: 3.6755 - accuracy: 0.7951
Epoch 74/200
2798/2798 [==============================] - 2s 819us/step - loss: 3.6818 - accuracy: 0.7985
Epoch 75/200
2798/2798 [==============================] - 2s 829us/step - loss: 3.6680 - accuracy: 0.7940
Epoch 76/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.6890 - accuracy: 0.7963
Epoch 77/200
2798/2798 [==============================] - 2s 825us/step - loss: 3.6709 - accuracy: 0.7942
Epoch 78/200
2798/2798 [==============================] - 2s 822us/step - loss: 3.6565 - accuracy: 0.7966
Epoch 79/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.6690 - accuracy: 0.7935
Epoch 80/200
2798/2798 [==============================] - 2s 817us/step - loss: 3.6537 - accuracy: 0.7966
Epoch 81/200
2798/2798 [==============================] - 2s 818us/step - loss: 3.6505 - accuracy: 0.7969
Epoch 82/200
2798/2798 [==============================] - 2s 837us/step - loss: 3.6095 - accuracy: 0.7978
Epoch 83/200
2798/2798 [==============================] - 2s 840us/step - loss: 3.6154 - accuracy: 0.7977
Epoch 84/200
2798/2798 [==============================] - 2s 848us/step - loss: 3.7035 - accuracy: 0.7983
Epoch 85/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.6720 - accuracy: 0.7972
Epoch 86/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.6538 - accuracy: 0.7964
Epoch 87/200
2798/2798 [==============================] - 2s 832us/step - loss: 3.6150 - accuracy: 0.7966
Epoch 88/200
2798/2798 [==============================] - 2s 822us/step - loss: 3.6640 - accuracy: 0.7975
Epoch 89/200
2798/2798 [==============================] - 2s 820us/step - loss: 3.6762 - accuracy: 0.7980
Epoch 90/200
2798/2798 [==============================] - 2s 820us/step - loss: 3.5972 - accuracy: 0.7974
Epoch 91/200
2798/2798 [==============================] - 2s 832us/step - loss: 3.6383 - accuracy: 0.7980
Epoch 92/200
2798/2798 [==============================] - 2s 828us/step - loss: 3.6101 - accuracy: 0.7981
Epoch 93/200
2798/2798 [==============================] - 2s 831us/step - loss: 3.6408 - accuracy: 0.7993
Epoch 94/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.5771 - accuracy: 0.8001
Epoch 95/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.6667 - accuracy: 0.7982
Epoch 96/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.5918 - accuracy: 0.8030
Epoch 97/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.6444 - accuracy: 0.7974
Epoch 98/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.6487 - accuracy: 0.7980
Epoch 99/200
2798/2798 [==============================] - 2s 817us/step - loss: 3.6028 - accuracy: 0.7990
Epoch 100/200
2798/2798 [==============================] - 2s 822us/step - loss: 3.6448 - accuracy: 0.7963
Epoch 101/200
2798/2798 [==============================] - 2s 820us/step - loss: 3.6125 - accuracy: 0.8005
Epoch 102/200
2798/2798 [==============================] - 2s 828us/step - loss: 3.6558 - accuracy: 0.8004
Epoch 103/200
2798/2798 [==============================] - 2s 830us/step - loss: 3.6060 - accuracy: 0.7983
Epoch 104/200
2798/2798 [==============================] - 2s 829us/step - loss: 3.5866 - accuracy: 0.7994
Epoch 105/200
2798/2798 [==============================] - 2s 831us/step - loss: 3.6322 - accuracy: 0.8014
Epoch 106/200
2798/2798 [==============================] - 2s 825us/step - loss: 3.6145 - accuracy: 0.7995
Epoch 107/200
2798/2798 [==============================] - 2s 842us/step - loss: 3.5540 - accuracy: 0.8006
Epoch 108/200
2798/2798 [==============================] - 2s 820us/step - loss: 3.6281 - accuracy: 0.7971
Epoch 109/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.5477 - accuracy: 0.8001
Epoch 110/200
2798/2798 [==============================] - 2s 817us/step - loss: 3.5837 - accuracy: 0.8025
Epoch 111/200
2798/2798 [==============================] - 2s 817us/step - loss: 3.5420 - accuracy: 0.8022
Epoch 112/200
2798/2798 [==============================] - 2s 820us/step - loss: 3.6077 - accuracy: 0.8018
Epoch 113/200
2798/2798 [==============================] - 2s 822us/step - loss: 3.5729 - accuracy: 0.8028
Epoch 114/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.5806 - accuracy: 0.8027
Epoch 115/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.5939 - accuracy: 0.8012
Epoch 116/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.5534 - accuracy: 0.8027
Epoch 117/200
2798/2798 [==============================] - 2s 819us/step - loss: 3.6042 - accuracy: 0.7960
Epoch 118/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.6079 - accuracy: 0.8004
Epoch 119/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.5741 - accuracy: 0.8018
Epoch 120/200
2798/2798 [==============================] - 2s 830us/step - loss: 3.5368 - accuracy: 0.8030
Epoch 121/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.5793 - accuracy: 0.8033
Epoch 122/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.5760 - accuracy: 0.8016
Epoch 123/200
2798/2798 [==============================] - 2s 819us/step - loss: 3.5927 - accuracy: 0.8013
Epoch 124/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.5653 - accuracy: 0.8030
Epoch 125/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.6106 - accuracy: 0.8016
Epoch 126/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.5710 - accuracy: 0.7992
Epoch 127/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.5565 - accuracy: 0.8020
Epoch 128/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.5909 - accuracy: 0.7999
Epoch 129/200
2798/2798 [==============================] - 2s 825us/step - loss: 3.5667 - accuracy: 0.8027
Epoch 130/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.6024 - accuracy: 0.8012
Epoch 131/200
2798/2798 [==============================] - 2s 822us/step - loss: 3.5801 - accuracy: 0.8014
Epoch 132/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.5303 - accuracy: 0.8030
Epoch 133/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.5346 - accuracy: 0.8029
Epoch 134/200
2798/2798 [==============================] - 2s 829us/step - loss: 3.6237 - accuracy: 0.8015
Epoch 135/200
2798/2798 [==============================] - 2s 826us/step - loss: 3.5511 - accuracy: 0.8018
Epoch 136/200
2798/2798 [==============================] - 2s 825us/step - loss: 3.5146 - accuracy: 0.8036
Epoch 137/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.4879 - accuracy: 0.8039
Epoch 138/200
2798/2798 [==============================] - 2s 829us/step - loss: 3.5399 - accuracy: 0.8041
Epoch 139/200
2798/2798 [==============================] - 2s 825us/step - loss: 3.4977 - accuracy: 0.8024
Epoch 140/200
2798/2798 [==============================] - 2s 817us/step - loss: 3.5492 - accuracy: 0.8037
Epoch 141/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.5264 - accuracy: 0.8012
Epoch 142/200
2798/2798 [==============================] - 2s 822us/step - loss: 3.5526 - accuracy: 0.8019
Epoch 143/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.5196 - accuracy: 0.8051
Epoch 144/200
2798/2798 [==============================] - 2s 826us/step - loss: 3.5304 - accuracy: 0.8002
Epoch 145/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.6259 - accuracy: 0.8012
Epoch 146/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.4732 - accuracy: 0.8058
Epoch 147/200
2798/2798 [==============================] - 2s 822us/step - loss: 3.5544 - accuracy: 0.8038
Epoch 148/200
2798/2798 [==============================] - 2s 822us/step - loss: 3.5544 - accuracy: 0.8030
Epoch 149/200
2798/2798 [==============================] - 2s 826us/step - loss: 3.5254 - accuracy: 0.8054
Epoch 150/200
2798/2798 [==============================] - 2s 825us/step - loss: 3.5470 - accuracy: 0.8033
Epoch 151/200
2798/2798 [==============================] - 2s 820us/step - loss: 3.5567 - accuracy: 0.8066
Epoch 152/200
2798/2798 [==============================] - 2s 828us/step - loss: 3.5443 - accuracy: 0.8048
Epoch 153/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.5805 - accuracy: 0.8025
Epoch 154/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.5086 - accuracy: 0.8055
Epoch 155/200
2798/2798 [==============================] - 2s 826us/step - loss: 3.4818 - accuracy: 0.8041
Epoch 156/200
2798/2798 [==============================] - 2s 825us/step - loss: 3.5338 - accuracy: 0.8047
Epoch 157/200
2798/2798 [==============================] - 2s 825us/step - loss: 3.5404 - accuracy: 0.8068
Epoch 158/200
2798/2798 [==============================] - 2s 833us/step - loss: 3.5665 - accuracy: 0.8039
Epoch 159/200
2798/2798 [==============================] - 2s 818us/step - loss: 3.5065 - accuracy: 0.8045
Epoch 160/200
2798/2798 [==============================] - 2s 838us/step - loss: 3.5396 - accuracy: 0.8012
Epoch 161/200
2798/2798 [==============================] - 2s 848us/step - loss: 3.5361 - accuracy: 0.8043
Epoch 162/200
2798/2798 [==============================] - 2s 841us/step - loss: 3.4955 - accuracy: 0.8064
Epoch 163/200
2798/2798 [==============================] - 2s 825us/step - loss: 3.5036 - accuracy: 0.8041
Epoch 164/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.4917 - accuracy: 0.8052
Epoch 165/200
2798/2798 [==============================] - 2s 823us/step - loss: 3.5625 - accuracy: 0.8039
Epoch 166/200
2798/2798 [==============================] - 2s 826us/step - loss: 3.5434 - accuracy: 0.8045
Epoch 167/200
2798/2798 [==============================] - 2s 819us/step - loss: 3.5117 - accuracy: 0.8041
Epoch 168/200
2798/2798 [==============================] - 2s 826us/step - loss: 3.4830 - accuracy: 0.8040
Epoch 169/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.5117 - accuracy: 0.8052
Epoch 170/200
2798/2798 [==============================] - 2s 826us/step - loss: 3.4734 - accuracy: 0.8047
Epoch 171/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.5371 - accuracy: 0.8048
Epoch 172/200
2798/2798 [==============================] - 2s 833us/step - loss: 3.5140 - accuracy: 0.8049
Epoch 173/200
2798/2798 [==============================] - 2s 833us/step - loss: 3.5219 - accuracy: 0.8051
Epoch 174/200
2798/2798 [==============================] - 2s 831us/step - loss: 3.5130 - accuracy: 0.8038
Epoch 175/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.5022 - accuracy: 0.8041
Epoch 176/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.5244 - accuracy: 0.8058
Epoch 177/200
2798/2798 [==============================] - 2s 819us/step - loss: 3.5246 - accuracy: 0.8055
Epoch 178/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.5209 - accuracy: 0.8040
Epoch 179/200
2798/2798 [==============================] - 2s 822us/step - loss: 3.5019 - accuracy: 0.8030
Epoch 180/200
2798/2798 [==============================] - 2s 816us/step - loss: 3.5052 - accuracy: 0.8052
Epoch 181/200
2798/2798 [==============================] - 2s 819us/step - loss: 3.5514 - accuracy: 0.8041
Epoch 182/200
2798/2798 [==============================] - 2s 828us/step - loss: 3.4876 - accuracy: 0.8060
Epoch 183/200
2798/2798 [==============================] - 2s 825us/step - loss: 3.5118 - accuracy: 0.8039
Epoch 184/200
2798/2798 [==============================] - 2s 815us/step - loss: 3.5713 - accuracy: 0.8058
Epoch 185/200
2798/2798 [==============================] - 2s 822us/step - loss: 3.5027 - accuracy: 0.8028
Epoch 186/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.5430 - accuracy: 0.8016
Epoch 187/200
2798/2798 [==============================] - 2s 828us/step - loss: 3.5255 - accuracy: 0.8061
Epoch 188/200
2798/2798 [==============================] - 2s 820us/step - loss: 3.5284 - accuracy: 0.8037
Epoch 189/200
2798/2798 [==============================] - 2s 825us/step - loss: 3.4539 - accuracy: 0.8070
Epoch 190/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.5292 - accuracy: 0.8058
Epoch 191/200
2798/2798 [==============================] - 2s 822us/step - loss: 3.5021 - accuracy: 0.8063
Epoch 192/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.4952 - accuracy: 0.8063
Epoch 193/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.5251 - accuracy: 0.8057
Epoch 194/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.4826 - accuracy: 0.8051
Epoch 195/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.5046 - accuracy: 0.8050
Epoch 196/200
2798/2798 [==============================] - 2s 825us/step - loss: 3.5152 - accuracy: 0.8042
Epoch 197/200
2798/2798 [==============================] - 2s 824us/step - loss: 3.4927 - accuracy: 0.8068
Epoch 198/200
2798/2798 [==============================] - 2s 822us/step - loss: 3.5331 - accuracy: 0.8051
Epoch 199/200
2798/2798 [==============================] - 2s 821us/step - loss: 3.4885 - accuracy: 0.8081
Epoch 200/200
2798/2798 [==============================] - 2s 827us/step - loss: 3.4917 - accuracy: 0.8045

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_base_200", model=model, model_history=history)

Observations

Despite 200 epochs of training, the neural network's accuracy is gradually improving in small increments. To further improve its performance, one option is to increase the complexity of the model architecture by incorporating additional layers and neurons.



Model Building with multiple layers and neurons (2-1) and 50 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(2, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=50, verbose=1)
Epoch 1/50
2798/2798 [==============================] - 3s 954us/step - loss: 50.6170 - accuracy: 0.3177
Epoch 2/50
2798/2798 [==============================] - 3s 956us/step - loss: 2.6792 - accuracy: 0.6000
Epoch 3/50
2798/2798 [==============================] - 3s 943us/step - loss: 1.1750 - accuracy: 0.7520
Epoch 4/50
2798/2798 [==============================] - 3s 960us/step - loss: 1.1521 - accuracy: 0.7751
Epoch 5/50
2798/2798 [==============================] - 3s 958us/step - loss: 1.0369 - accuracy: 0.7884
Epoch 6/50
2798/2798 [==============================] - 3s 955us/step - loss: 1.0123 - accuracy: 0.8005
Epoch 7/50
2798/2798 [==============================] - 3s 961us/step - loss: 0.9853 - accuracy: 0.8048
Epoch 8/50
2798/2798 [==============================] - 3s 947us/step - loss: 0.9137 - accuracy: 0.8121
Epoch 9/50
2798/2798 [==============================] - 3s 948us/step - loss: 0.9651 - accuracy: 0.8143
Epoch 10/50
2798/2798 [==============================] - 3s 957us/step - loss: 0.9051 - accuracy: 0.8186
Epoch 11/50
2798/2798 [==============================] - 3s 966us/step - loss: 0.8560 - accuracy: 0.8193
Epoch 12/50
2798/2798 [==============================] - 3s 950us/step - loss: 0.7790 - accuracy: 0.8314
Epoch 13/50
2798/2798 [==============================] - 3s 949us/step - loss: 0.9530 - accuracy: 0.8176
Epoch 14/50
2798/2798 [==============================] - 3s 948us/step - loss: 0.8531 - accuracy: 0.8276
Epoch 15/50
2798/2798 [==============================] - 3s 941us/step - loss: 0.9158 - accuracy: 0.8243
Epoch 16/50
2798/2798 [==============================] - 3s 950us/step - loss: 0.8231 - accuracy: 0.8281
Epoch 17/50
2798/2798 [==============================] - 3s 940us/step - loss: 0.8327 - accuracy: 0.8325
Epoch 18/50
2798/2798 [==============================] - 3s 950us/step - loss: 0.7618 - accuracy: 0.8379
Epoch 19/50
2798/2798 [==============================] - 3s 961us/step - loss: 0.8297 - accuracy: 0.8334
Epoch 20/50
2798/2798 [==============================] - 3s 958us/step - loss: 0.8035 - accuracy: 0.8372
Epoch 21/50
2798/2798 [==============================] - 3s 946us/step - loss: 0.7906 - accuracy: 0.8357
Epoch 22/50
2798/2798 [==============================] - 3s 961us/step - loss: 0.8064 - accuracy: 0.8364
Epoch 23/50
2798/2798 [==============================] - 3s 950us/step - loss: 0.7690 - accuracy: 0.8414
Epoch 24/50
2798/2798 [==============================] - 3s 954us/step - loss: 0.7835 - accuracy: 0.8401
Epoch 25/50
2798/2798 [==============================] - 3s 955us/step - loss: 0.7585 - accuracy: 0.8462
Epoch 26/50
2798/2798 [==============================] - 3s 959us/step - loss: 0.7425 - accuracy: 0.8474
Epoch 27/50
2798/2798 [==============================] - 3s 942us/step - loss: 0.8337 - accuracy: 0.8353
Epoch 28/50
2798/2798 [==============================] - 3s 937us/step - loss: 0.7505 - accuracy: 0.8488
Epoch 29/50
2798/2798 [==============================] - 3s 956us/step - loss: 0.7463 - accuracy: 0.8490
Epoch 30/50
2798/2798 [==============================] - 3s 955us/step - loss: 0.7365 - accuracy: 0.8496
Epoch 31/50
2798/2798 [==============================] - 3s 953us/step - loss: 0.7725 - accuracy: 0.8467
Epoch 32/50
2798/2798 [==============================] - 3s 971us/step - loss: 0.7010 - accuracy: 0.8538
Epoch 33/50
2798/2798 [==============================] - 3s 992us/step - loss: 0.7113 - accuracy: 0.8524
Epoch 34/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8191 - accuracy: 0.8436
Epoch 35/50
2798/2798 [==============================] - 3s 957us/step - loss: 0.7897 - accuracy: 0.8489
Epoch 36/50
2798/2798 [==============================] - 3s 965us/step - loss: 0.6759 - accuracy: 0.8596
Epoch 37/50
2798/2798 [==============================] - 3s 958us/step - loss: 0.7138 - accuracy: 0.8522
Epoch 38/50
2798/2798 [==============================] - 3s 950us/step - loss: 0.7806 - accuracy: 0.8495
Epoch 39/50
2798/2798 [==============================] - 3s 961us/step - loss: 0.7358 - accuracy: 0.8542
Epoch 40/50
2798/2798 [==============================] - 3s 960us/step - loss: 0.7005 - accuracy: 0.8549
Epoch 41/50
2798/2798 [==============================] - 3s 949us/step - loss: 0.7454 - accuracy: 0.8539
Epoch 42/50
2798/2798 [==============================] - 3s 942us/step - loss: 0.6653 - accuracy: 0.8602
Epoch 43/50
2798/2798 [==============================] - 3s 949us/step - loss: 0.7194 - accuracy: 0.8579
Epoch 44/50
2798/2798 [==============================] - 3s 960us/step - loss: 0.6678 - accuracy: 0.8617
Epoch 45/50
2798/2798 [==============================] - 3s 954us/step - loss: 0.7029 - accuracy: 0.8615
Epoch 46/50
2798/2798 [==============================] - 3s 952us/step - loss: 0.6772 - accuracy: 0.8635
Epoch 47/50
2798/2798 [==============================] - 3s 961us/step - loss: 0.6554 - accuracy: 0.8621
Epoch 48/50
2798/2798 [==============================] - 3s 951us/step - loss: 0.6603 - accuracy: 0.8629
Epoch 49/50
2798/2798 [==============================] - 3s 937us/step - loss: 0.6439 - accuracy: 0.8665
Epoch 50/50
2798/2798 [==============================] - 3s 955us/step - loss: 0.6499 - accuracy: 0.8630

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_2_1_50_epochs", model=model, model_history=history)

Observations

The current neural network architecture has two layers with 2 and 1 neurons in each layer, and it is not showing signs of overfitting with 50 epochs. Therefore, it may be worth considering increasing the size of the sequential model network or number of epochs to potentially improve its performance further.



Model Building with multiple layers and neurons (2-1) and 100 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(2, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=100, verbose=1)
Epoch 1/100
2798/2798 [==============================] - 3s 964us/step - loss: 230.6059 - accuracy: 0.3140
Epoch 2/100
2798/2798 [==============================] - 3s 953us/step - loss: 20.3735 - accuracy: 0.3863
Epoch 3/100
2798/2798 [==============================] - 3s 962us/step - loss: 3.5473 - accuracy: 0.6936
Epoch 4/100
2798/2798 [==============================] - 3s 956us/step - loss: 2.7848 - accuracy: 0.7584
Epoch 5/100
2798/2798 [==============================] - 3s 946us/step - loss: 2.5332 - accuracy: 0.7713
Epoch 6/100
2798/2798 [==============================] - 3s 957us/step - loss: 2.1089 - accuracy: 0.7871
Epoch 7/100
2798/2798 [==============================] - 3s 955us/step - loss: 1.8596 - accuracy: 0.7991
Epoch 8/100
2798/2798 [==============================] - 3s 965us/step - loss: 1.9122 - accuracy: 0.8026
Epoch 9/100
2798/2798 [==============================] - 3s 951us/step - loss: 1.7837 - accuracy: 0.8040
Epoch 10/100
2798/2798 [==============================] - 3s 950us/step - loss: 1.8104 - accuracy: 0.8071
Epoch 11/100
2798/2798 [==============================] - 3s 951us/step - loss: 1.6307 - accuracy: 0.8100
Epoch 12/100
2798/2798 [==============================] - 3s 957us/step - loss: 1.7026 - accuracy: 0.8071
Epoch 13/100
2798/2798 [==============================] - 3s 956us/step - loss: 1.6557 - accuracy: 0.8091
Epoch 14/100
2798/2798 [==============================] - 3s 958us/step - loss: 1.6256 - accuracy: 0.8109
Epoch 15/100
2798/2798 [==============================] - 3s 947us/step - loss: 1.5361 - accuracy: 0.8152
Epoch 16/100
2798/2798 [==============================] - 3s 954us/step - loss: 1.5131 - accuracy: 0.8105
Epoch 17/100
2798/2798 [==============================] - 3s 957us/step - loss: 1.4541 - accuracy: 0.8124
Epoch 18/100
2798/2798 [==============================] - 3s 954us/step - loss: 1.7812 - accuracy: 0.8042
Epoch 19/100
2798/2798 [==============================] - 3s 963us/step - loss: 1.4080 - accuracy: 0.8101
Epoch 20/100
2798/2798 [==============================] - 3s 937us/step - loss: 1.5224 - accuracy: 0.8070
Epoch 21/100
2798/2798 [==============================] - 3s 949us/step - loss: 1.5642 - accuracy: 0.8093
Epoch 22/100
2798/2798 [==============================] - 3s 951us/step - loss: 1.5350 - accuracy: 0.8056
Epoch 23/100
2798/2798 [==============================] - 3s 951us/step - loss: 1.4434 - accuracy: 0.8188
Epoch 24/100
2798/2798 [==============================] - 3s 954us/step - loss: 1.5598 - accuracy: 0.8084
Epoch 25/100
2798/2798 [==============================] - 3s 958us/step - loss: 1.6324 - accuracy: 0.8095
Epoch 26/100
2798/2798 [==============================] - 3s 947us/step - loss: 1.5171 - accuracy: 0.8110
Epoch 27/100
2798/2798 [==============================] - 3s 962us/step - loss: 1.5239 - accuracy: 0.8085
Epoch 28/100
2798/2798 [==============================] - 3s 963us/step - loss: 1.5262 - accuracy: 0.8099
Epoch 29/100
2798/2798 [==============================] - 3s 964us/step - loss: 1.5208 - accuracy: 0.8101
Epoch 30/100
2798/2798 [==============================] - 3s 952us/step - loss: 1.5820 - accuracy: 0.8109
Epoch 31/100
2798/2798 [==============================] - 3s 959us/step - loss: 1.5384 - accuracy: 0.8128
Epoch 32/100
2798/2798 [==============================] - 3s 963us/step - loss: 1.5913 - accuracy: 0.8087
Epoch 33/100
2798/2798 [==============================] - 3s 968us/step - loss: 1.5596 - accuracy: 0.8121
Epoch 34/100
2798/2798 [==============================] - 3s 968us/step - loss: 1.4636 - accuracy: 0.8097
Epoch 35/100
2798/2798 [==============================] - 3s 953us/step - loss: 1.4639 - accuracy: 0.8147
Epoch 36/100
2798/2798 [==============================] - 3s 966us/step - loss: 1.5050 - accuracy: 0.8141
Epoch 37/100
2798/2798 [==============================] - 3s 958us/step - loss: 1.3776 - accuracy: 0.8186
Epoch 38/100
2798/2798 [==============================] - 3s 962us/step - loss: 1.4876 - accuracy: 0.8140
Epoch 39/100
2798/2798 [==============================] - 3s 943us/step - loss: 1.5122 - accuracy: 0.8112
Epoch 40/100
2798/2798 [==============================] - 3s 951us/step - loss: 1.4486 - accuracy: 0.8154
Epoch 41/100
2798/2798 [==============================] - 3s 966us/step - loss: 1.5004 - accuracy: 0.8161
Epoch 42/100
2798/2798 [==============================] - 3s 958us/step - loss: 1.4580 - accuracy: 0.8153
Epoch 43/100
2798/2798 [==============================] - 3s 953us/step - loss: 1.4579 - accuracy: 0.8142
Epoch 44/100
2798/2798 [==============================] - 3s 992us/step - loss: 1.4352 - accuracy: 0.8172
Epoch 45/100
2798/2798 [==============================] - 3s 1ms/step - loss: 1.3910 - accuracy: 0.8162
Epoch 46/100
2798/2798 [==============================] - 3s 959us/step - loss: 1.4093 - accuracy: 0.8155
Epoch 47/100
2798/2798 [==============================] - 3s 948us/step - loss: 1.5400 - accuracy: 0.8104
Epoch 48/100
2798/2798 [==============================] - 3s 953us/step - loss: 1.4033 - accuracy: 0.8186
Epoch 49/100
2798/2798 [==============================] - 3s 961us/step - loss: 1.5048 - accuracy: 0.8136
Epoch 50/100
2798/2798 [==============================] - 3s 949us/step - loss: 1.4010 - accuracy: 0.8169
Epoch 51/100
2798/2798 [==============================] - 3s 940us/step - loss: 1.4677 - accuracy: 0.8138
Epoch 52/100
2798/2798 [==============================] - 3s 948us/step - loss: 1.4679 - accuracy: 0.8140
Epoch 53/100
2798/2798 [==============================] - 3s 950us/step - loss: 1.3713 - accuracy: 0.8137
Epoch 54/100
2798/2798 [==============================] - 3s 942us/step - loss: 1.4481 - accuracy: 0.8151
Epoch 55/100
2798/2798 [==============================] - 3s 943us/step - loss: 1.3604 - accuracy: 0.8148
Epoch 56/100
2798/2798 [==============================] - 3s 956us/step - loss: 1.4153 - accuracy: 0.8141
Epoch 57/100
2798/2798 [==============================] - 3s 945us/step - loss: 1.3326 - accuracy: 0.8198
Epoch 58/100
2798/2798 [==============================] - 3s 944us/step - loss: 1.3178 - accuracy: 0.8171
Epoch 59/100
2798/2798 [==============================] - 3s 943us/step - loss: 1.4193 - accuracy: 0.8195
Epoch 60/100
2798/2798 [==============================] - 3s 951us/step - loss: 1.2555 - accuracy: 0.8224
Epoch 61/100
2798/2798 [==============================] - 3s 957us/step - loss: 1.4039 - accuracy: 0.8156
Epoch 62/100
2798/2798 [==============================] - 3s 947us/step - loss: 1.5110 - accuracy: 0.8124
Epoch 63/100
2798/2798 [==============================] - 3s 951us/step - loss: 1.3393 - accuracy: 0.8186
Epoch 64/100
2798/2798 [==============================] - 3s 957us/step - loss: 1.2916 - accuracy: 0.8218
Epoch 65/100
2798/2798 [==============================] - 3s 951us/step - loss: 1.2841 - accuracy: 0.8182
Epoch 66/100
2798/2798 [==============================] - 3s 961us/step - loss: 1.4162 - accuracy: 0.8141
Epoch 67/100
2798/2798 [==============================] - 3s 970us/step - loss: 1.2352 - accuracy: 0.8208
Epoch 68/100
2798/2798 [==============================] - 3s 960us/step - loss: 1.3526 - accuracy: 0.8167
Epoch 69/100
2798/2798 [==============================] - 3s 957us/step - loss: 1.3350 - accuracy: 0.8188
Epoch 70/100
2798/2798 [==============================] - 3s 955us/step - loss: 1.3107 - accuracy: 0.8176
Epoch 71/100
2798/2798 [==============================] - 3s 955us/step - loss: 1.3025 - accuracy: 0.8180
Epoch 72/100
2798/2798 [==============================] - 3s 954us/step - loss: 1.3005 - accuracy: 0.8182
Epoch 73/100
2798/2798 [==============================] - 3s 947us/step - loss: 1.2769 - accuracy: 0.8168
Epoch 74/100
2798/2798 [==============================] - 3s 959us/step - loss: 1.2503 - accuracy: 0.8204
Epoch 75/100
2798/2798 [==============================] - 3s 956us/step - loss: 1.2820 - accuracy: 0.8189
Epoch 76/100
2798/2798 [==============================] - 3s 954us/step - loss: 1.2341 - accuracy: 0.8186
Epoch 77/100
2798/2798 [==============================] - 3s 946us/step - loss: 1.3054 - accuracy: 0.8183
Epoch 78/100
2798/2798 [==============================] - 3s 950us/step - loss: 1.2250 - accuracy: 0.8189
Epoch 79/100
2798/2798 [==============================] - 3s 955us/step - loss: 1.1838 - accuracy: 0.8220
Epoch 80/100
2798/2798 [==============================] - 3s 944us/step - loss: 1.2336 - accuracy: 0.8195
Epoch 81/100
2798/2798 [==============================] - 3s 933us/step - loss: 1.2295 - accuracy: 0.8183
Epoch 82/100
2798/2798 [==============================] - 3s 944us/step - loss: 1.2303 - accuracy: 0.8184
Epoch 83/100
2798/2798 [==============================] - 3s 954us/step - loss: 1.2269 - accuracy: 0.8203
Epoch 84/100
2798/2798 [==============================] - 3s 948us/step - loss: 1.1600 - accuracy: 0.8190
Epoch 85/100
2798/2798 [==============================] - 3s 963us/step - loss: 1.1882 - accuracy: 0.8208
Epoch 86/100
2798/2798 [==============================] - 3s 943us/step - loss: 1.2756 - accuracy: 0.8163
Epoch 87/100
2798/2798 [==============================] - 3s 952us/step - loss: 1.1152 - accuracy: 0.8205
Epoch 88/100
2798/2798 [==============================] - 3s 969us/step - loss: 1.1347 - accuracy: 0.8200
Epoch 89/100
2798/2798 [==============================] - 3s 966us/step - loss: 1.0911 - accuracy: 0.8222
Epoch 90/100
2798/2798 [==============================] - 3s 953us/step - loss: 1.1556 - accuracy: 0.8207
Epoch 91/100
2798/2798 [==============================] - 3s 956us/step - loss: 1.1264 - accuracy: 0.8209
Epoch 92/100
2798/2798 [==============================] - 3s 969us/step - loss: 1.1688 - accuracy: 0.8181
Epoch 93/100
2798/2798 [==============================] - 3s 967us/step - loss: 1.0943 - accuracy: 0.8214
Epoch 94/100
2798/2798 [==============================] - 3s 955us/step - loss: 1.1008 - accuracy: 0.8236
Epoch 95/100
2798/2798 [==============================] - 3s 947us/step - loss: 1.0978 - accuracy: 0.8229
Epoch 96/100
2798/2798 [==============================] - 3s 961us/step - loss: 1.1122 - accuracy: 0.8198
Epoch 97/100
2798/2798 [==============================] - 3s 948us/step - loss: 1.1482 - accuracy: 0.8200
Epoch 98/100
2798/2798 [==============================] - 3s 955us/step - loss: 1.0440 - accuracy: 0.8232
Epoch 99/100
2798/2798 [==============================] - 3s 950us/step - loss: 1.0031 - accuracy: 0.8263
Epoch 100/100
2798/2798 [==============================] - 3s 952us/step - loss: 1.0145 - accuracy: 0.8245

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_2_1_100_epochs", model=model, model_history=history)

Observations

The current neural network architecture has two layers with 2 and 1 neurons in each layer, and it is not showing signs of overfitting with 100 epochs. Therefore, it may be worth considering increasing the size of the sequential model network i.e to increase the number of neurons.



Model Building with multiple layers and neurons (4-1) and 50 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(4, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=50, verbose=1)
Epoch 1/50
2798/2798 [==============================] - 4s 970us/step - loss: 120.1924 - accuracy: 0.4141
Epoch 2/50
2798/2798 [==============================] - 3s 962us/step - loss: 1.7123 - accuracy: 0.8073
Epoch 3/50
2798/2798 [==============================] - 3s 965us/step - loss: 1.2568 - accuracy: 0.8480
Epoch 4/50
2798/2798 [==============================] - 3s 959us/step - loss: 1.1885 - accuracy: 0.8561
Epoch 5/50
2798/2798 [==============================] - 3s 969us/step - loss: 1.1438 - accuracy: 0.8585
Epoch 6/50
2798/2798 [==============================] - 3s 963us/step - loss: 1.0676 - accuracy: 0.8636
Epoch 7/50
2798/2798 [==============================] - 3s 965us/step - loss: 1.1185 - accuracy: 0.8596
Epoch 8/50
2798/2798 [==============================] - 3s 964us/step - loss: 1.0611 - accuracy: 0.8597
Epoch 9/50
2798/2798 [==============================] - 3s 959us/step - loss: 0.9655 - accuracy: 0.8579
Epoch 10/50
2798/2798 [==============================] - 3s 962us/step - loss: 0.9259 - accuracy: 0.8600
Epoch 11/50
2798/2798 [==============================] - 3s 947us/step - loss: 0.8930 - accuracy: 0.8617
Epoch 12/50
2798/2798 [==============================] - 3s 942us/step - loss: 0.8768 - accuracy: 0.8602
Epoch 13/50
2798/2798 [==============================] - 3s 968us/step - loss: 0.9166 - accuracy: 0.8571
Epoch 14/50
2798/2798 [==============================] - 3s 964us/step - loss: 0.8541 - accuracy: 0.8613
Epoch 15/50
2798/2798 [==============================] - 3s 961us/step - loss: 0.8697 - accuracy: 0.8621
Epoch 16/50
2798/2798 [==============================] - 3s 960us/step - loss: 0.9086 - accuracy: 0.8544
Epoch 17/50
2798/2798 [==============================] - 3s 956us/step - loss: 0.7969 - accuracy: 0.8663
Epoch 18/50
2798/2798 [==============================] - 3s 942us/step - loss: 0.7846 - accuracy: 0.8648
Epoch 19/50
2798/2798 [==============================] - 3s 960us/step - loss: 0.7549 - accuracy: 0.8648
Epoch 20/50
2798/2798 [==============================] - 3s 947us/step - loss: 0.7866 - accuracy: 0.8626
Epoch 21/50
2798/2798 [==============================] - 3s 946us/step - loss: 0.7918 - accuracy: 0.8624
Epoch 22/50
2798/2798 [==============================] - 3s 950us/step - loss: 0.7926 - accuracy: 0.8592
Epoch 23/50
2798/2798 [==============================] - 3s 966us/step - loss: 0.7547 - accuracy: 0.8647
Epoch 24/50
2798/2798 [==============================] - 3s 960us/step - loss: 0.7556 - accuracy: 0.8644
Epoch 25/50
2798/2798 [==============================] - 3s 953us/step - loss: 0.6925 - accuracy: 0.8673
Epoch 26/50
2798/2798 [==============================] - 3s 956us/step - loss: 0.6219 - accuracy: 0.9070
Epoch 27/50
2798/2798 [==============================] - 3s 963us/step - loss: 0.5469 - accuracy: 0.9166
Epoch 28/50
2798/2798 [==============================] - 3s 951us/step - loss: 0.5645 - accuracy: 0.9134
Epoch 29/50
2798/2798 [==============================] - 3s 966us/step - loss: 0.5292 - accuracy: 0.9169
Epoch 30/50
2798/2798 [==============================] - 3s 957us/step - loss: 0.5437 - accuracy: 0.9151
Epoch 31/50
2798/2798 [==============================] - 3s 965us/step - loss: 0.4876 - accuracy: 0.9200
Epoch 32/50
2798/2798 [==============================] - 3s 963us/step - loss: 0.5331 - accuracy: 0.9144
Epoch 33/50
2798/2798 [==============================] - 3s 961us/step - loss: 0.4974 - accuracy: 0.9172
Epoch 34/50
2798/2798 [==============================] - 3s 957us/step - loss: 0.5021 - accuracy: 0.9182
Epoch 35/50
2798/2798 [==============================] - 3s 959us/step - loss: 0.4941 - accuracy: 0.9196
Epoch 36/50
2798/2798 [==============================] - 3s 956us/step - loss: 0.4777 - accuracy: 0.9203
Epoch 37/50
2798/2798 [==============================] - 3s 968us/step - loss: 0.4529 - accuracy: 0.9222
Epoch 38/50
2798/2798 [==============================] - 3s 949us/step - loss: 0.4963 - accuracy: 0.9205
Epoch 39/50
2798/2798 [==============================] - 3s 956us/step - loss: 0.4615 - accuracy: 0.9221
Epoch 40/50
2798/2798 [==============================] - 3s 949us/step - loss: 0.4432 - accuracy: 0.9239
Epoch 41/50
2798/2798 [==============================] - 3s 949us/step - loss: 0.4707 - accuracy: 0.9211
Epoch 42/50
2798/2798 [==============================] - 3s 965us/step - loss: 0.4964 - accuracy: 0.9212
Epoch 43/50
2798/2798 [==============================] - 3s 967us/step - loss: 0.4589 - accuracy: 0.9209
Epoch 44/50
2798/2798 [==============================] - 3s 956us/step - loss: 0.4908 - accuracy: 0.9198
Epoch 45/50
2798/2798 [==============================] - 3s 960us/step - loss: 0.4279 - accuracy: 0.9276
Epoch 46/50
2798/2798 [==============================] - 3s 953us/step - loss: 0.3982 - accuracy: 0.9263
Epoch 47/50
2798/2798 [==============================] - 3s 965us/step - loss: 0.4502 - accuracy: 0.9221
Epoch 48/50
2798/2798 [==============================] - 3s 963us/step - loss: 0.4210 - accuracy: 0.9235
Epoch 49/50
2798/2798 [==============================] - 3s 964us/step - loss: 0.4185 - accuracy: 0.9240
Epoch 50/50
2798/2798 [==============================] - 3s 961us/step - loss: 0.3968 - accuracy: 0.9264

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_4_1_50_epochs", model=model, model_history=history)

Observations

The current neural network architecture has two layers with 2 and 1 neurons in each layer, and it is not showing signs of overfitting with 50 epochs. Therefore, it may be worth considering increasing the number of epochs to potentially improve its performance further.



Model Building with multiple layers and neurons (4-1) and 100 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(4, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=100, verbose=1)
Epoch 1/100
2798/2798 [==============================] - 3s 964us/step - loss: 99.1253 - accuracy: 0.4849
Epoch 2/100
2798/2798 [==============================] - 3s 953us/step - loss: 3.6486 - accuracy: 0.7797
Epoch 3/100
2798/2798 [==============================] - 3s 942us/step - loss: 1.4164 - accuracy: 0.8266
Epoch 4/100
2798/2798 [==============================] - 3s 951us/step - loss: 0.9565 - accuracy: 0.8432
Epoch 5/100
2798/2798 [==============================] - 3s 959us/step - loss: 1.0559 - accuracy: 0.8427
Epoch 6/100
2798/2798 [==============================] - 3s 951us/step - loss: 1.0512 - accuracy: 0.8430
Epoch 7/100
2798/2798 [==============================] - 3s 962us/step - loss: 1.0205 - accuracy: 0.8480
Epoch 8/100
2798/2798 [==============================] - 3s 944us/step - loss: 1.1111 - accuracy: 0.8474
Epoch 9/100
2798/2798 [==============================] - 3s 958us/step - loss: 0.9577 - accuracy: 0.8558
Epoch 10/100
2798/2798 [==============================] - 3s 957us/step - loss: 1.1330 - accuracy: 0.8475
Epoch 11/100
2798/2798 [==============================] - 3s 960us/step - loss: 1.0025 - accuracy: 0.8560
Epoch 12/100
2798/2798 [==============================] - 3s 962us/step - loss: 1.1878 - accuracy: 0.8468
Epoch 13/100
2798/2798 [==============================] - 3s 957us/step - loss: 0.9911 - accuracy: 0.8611
Epoch 14/100
2798/2798 [==============================] - 3s 973us/step - loss: 1.0127 - accuracy: 0.8567
Epoch 15/100
2798/2798 [==============================] - 3s 955us/step - loss: 1.0226 - accuracy: 0.8563
Epoch 16/100
2798/2798 [==============================] - 3s 961us/step - loss: 1.0021 - accuracy: 0.8603
Epoch 17/100
2798/2798 [==============================] - 3s 957us/step - loss: 0.9862 - accuracy: 0.8610
Epoch 18/100
2798/2798 [==============================] - 3s 953us/step - loss: 0.9789 - accuracy: 0.8627
Epoch 19/100
2798/2798 [==============================] - 3s 969us/step - loss: 1.0424 - accuracy: 0.8576
Epoch 20/100
2798/2798 [==============================] - 3s 954us/step - loss: 0.9977 - accuracy: 0.8596
Epoch 21/100
2798/2798 [==============================] - 3s 970us/step - loss: 1.0064 - accuracy: 0.8607
Epoch 22/100
2798/2798 [==============================] - 3s 965us/step - loss: 0.9634 - accuracy: 0.8643
Epoch 23/100
2798/2798 [==============================] - 3s 951us/step - loss: 0.9897 - accuracy: 0.8601
Epoch 24/100
2798/2798 [==============================] - 3s 940us/step - loss: 1.0323 - accuracy: 0.8605
Epoch 25/100
2798/2798 [==============================] - 3s 971us/step - loss: 0.9492 - accuracy: 0.8639
Epoch 26/100
2798/2798 [==============================] - 3s 969us/step - loss: 0.8974 - accuracy: 0.8655
Epoch 27/100
2798/2798 [==============================] - 3s 973us/step - loss: 0.9667 - accuracy: 0.8627
Epoch 28/100
2798/2798 [==============================] - 3s 964us/step - loss: 0.9231 - accuracy: 0.8643
Epoch 29/100
2798/2798 [==============================] - 3s 961us/step - loss: 0.9447 - accuracy: 0.8599
Epoch 30/100
2798/2798 [==============================] - 3s 959us/step - loss: 0.8968 - accuracy: 0.8630
Epoch 31/100
2798/2798 [==============================] - 3s 975us/step - loss: 0.9710 - accuracy: 0.8593
Epoch 32/100
2798/2798 [==============================] - 3s 949us/step - loss: 0.9536 - accuracy: 0.8588
Epoch 33/100
2798/2798 [==============================] - 3s 968us/step - loss: 0.9234 - accuracy: 0.8646
Epoch 34/100
2798/2798 [==============================] - 3s 961us/step - loss: 0.8870 - accuracy: 0.8649
Epoch 35/100
2798/2798 [==============================] - 3s 965us/step - loss: 0.9003 - accuracy: 0.8648
Epoch 36/100
2798/2798 [==============================] - 3s 963us/step - loss: 0.9137 - accuracy: 0.8636
Epoch 37/100
2798/2798 [==============================] - 3s 964us/step - loss: 0.9166 - accuracy: 0.8635
Epoch 38/100
2798/2798 [==============================] - 3s 965us/step - loss: 0.8611 - accuracy: 0.8646
Epoch 39/100
2798/2798 [==============================] - 3s 953us/step - loss: 0.8493 - accuracy: 0.8647
Epoch 40/100
2798/2798 [==============================] - 3s 970us/step - loss: 0.8635 - accuracy: 0.8643
Epoch 41/100
2798/2798 [==============================] - 3s 968us/step - loss: 0.9292 - accuracy: 0.8600
Epoch 42/100
2798/2798 [==============================] - 3s 989us/step - loss: 0.8050 - accuracy: 0.8666
Epoch 43/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8002 - accuracy: 0.8676
Epoch 44/100
2798/2798 [==============================] - 3s 992us/step - loss: 0.8544 - accuracy: 0.8621
Epoch 45/100
2798/2798 [==============================] - 3s 967us/step - loss: 0.8557 - accuracy: 0.8626
Epoch 46/100
2798/2798 [==============================] - 3s 971us/step - loss: 0.7600 - accuracy: 0.8660
Epoch 47/100
2798/2798 [==============================] - 3s 967us/step - loss: 0.7835 - accuracy: 0.8666
Epoch 48/100
2798/2798 [==============================] - 3s 967us/step - loss: 0.8156 - accuracy: 0.8651
Epoch 49/100
2798/2798 [==============================] - 3s 971us/step - loss: 0.7871 - accuracy: 0.8675
Epoch 50/100
2798/2798 [==============================] - 3s 967us/step - loss: 0.8364 - accuracy: 0.8613
Epoch 51/100
2798/2798 [==============================] - 3s 971us/step - loss: 0.7776 - accuracy: 0.8689
Epoch 52/100
2798/2798 [==============================] - 3s 955us/step - loss: 0.7181 - accuracy: 0.8692
Epoch 53/100
2798/2798 [==============================] - 3s 959us/step - loss: 0.7550 - accuracy: 0.8655
Epoch 54/100
2798/2798 [==============================] - 3s 954us/step - loss: 0.7911 - accuracy: 0.8645
Epoch 55/100
2798/2798 [==============================] - 3s 971us/step - loss: 0.8181 - accuracy: 0.8593
Epoch 56/100
2798/2798 [==============================] - 3s 949us/step - loss: 0.7398 - accuracy: 0.8642
Epoch 57/100
2798/2798 [==============================] - 3s 963us/step - loss: 0.7260 - accuracy: 0.8702
Epoch 58/100
2798/2798 [==============================] - 3s 962us/step - loss: 0.7404 - accuracy: 0.8693
Epoch 59/100
2798/2798 [==============================] - 3s 952us/step - loss: 0.7257 - accuracy: 0.8650
Epoch 60/100
2798/2798 [==============================] - 3s 963us/step - loss: 0.7583 - accuracy: 0.8627
Epoch 61/100
2798/2798 [==============================] - 3s 969us/step - loss: 0.6859 - accuracy: 0.8679
Epoch 62/100
2798/2798 [==============================] - 3s 960us/step - loss: 0.7583 - accuracy: 0.8632
Epoch 63/100
2798/2798 [==============================] - 3s 964us/step - loss: 0.6320 - accuracy: 0.8732
Epoch 64/100
2798/2798 [==============================] - 3s 961us/step - loss: 0.7050 - accuracy: 0.8657
Epoch 65/100
2798/2798 [==============================] - 3s 951us/step - loss: 0.7059 - accuracy: 0.8675
Epoch 66/100
2798/2798 [==============================] - 3s 947us/step - loss: 0.6384 - accuracy: 0.8707
Epoch 67/100
2798/2798 [==============================] - 3s 959us/step - loss: 0.6618 - accuracy: 0.8672
Epoch 68/100
2798/2798 [==============================] - 3s 954us/step - loss: 0.6569 - accuracy: 0.8710
Epoch 69/100
2798/2798 [==============================] - 3s 965us/step - loss: 0.6622 - accuracy: 0.8664
Epoch 70/100
2798/2798 [==============================] - 3s 958us/step - loss: 0.5954 - accuracy: 0.8728
Epoch 71/100
2798/2798 [==============================] - 3s 959us/step - loss: 0.5721 - accuracy: 0.8704
Epoch 72/100
2798/2798 [==============================] - 3s 963us/step - loss: 0.5814 - accuracy: 0.8730
Epoch 73/100
2798/2798 [==============================] - 3s 967us/step - loss: 0.5923 - accuracy: 0.8722
Epoch 74/100
2798/2798 [==============================] - 3s 970us/step - loss: 0.6024 - accuracy: 0.8675
Epoch 75/100
2798/2798 [==============================] - 3s 967us/step - loss: 0.6095 - accuracy: 0.8686
Epoch 76/100
2798/2798 [==============================] - 3s 959us/step - loss: 0.5680 - accuracy: 0.8718
Epoch 77/100
2798/2798 [==============================] - 3s 968us/step - loss: 0.5829 - accuracy: 0.8700
Epoch 78/100
2798/2798 [==============================] - 3s 965us/step - loss: 0.6071 - accuracy: 0.8639
Epoch 79/100
2798/2798 [==============================] - 3s 954us/step - loss: 0.5946 - accuracy: 0.8701
Epoch 80/100
2798/2798 [==============================] - 3s 958us/step - loss: 0.5368 - accuracy: 0.8737
Epoch 81/100
2798/2798 [==============================] - 3s 975us/step - loss: 0.4853 - accuracy: 0.8743
Epoch 82/100
2798/2798 [==============================] - 3s 950us/step - loss: 0.4613 - accuracy: 0.8739
Epoch 83/100
2798/2798 [==============================] - 3s 965us/step - loss: 0.4811 - accuracy: 0.8713
Epoch 84/100
2798/2798 [==============================] - 3s 952us/step - loss: 0.4499 - accuracy: 0.8728
Epoch 85/100
2798/2798 [==============================] - 3s 967us/step - loss: 0.4552 - accuracy: 0.8703
Epoch 86/100
2798/2798 [==============================] - 3s 951us/step - loss: 0.4486 - accuracy: 0.8717
Epoch 87/100
2798/2798 [==============================] - 3s 949us/step - loss: 0.4241 - accuracy: 0.8740
Epoch 88/100
2798/2798 [==============================] - 3s 963us/step - loss: 0.4425 - accuracy: 0.8712
Epoch 89/100
2798/2798 [==============================] - 3s 971us/step - loss: 0.4187 - accuracy: 0.8819
Epoch 90/100
2798/2798 [==============================] - 3s 966us/step - loss: 0.4281 - accuracy: 0.8844
Epoch 91/100
2798/2798 [==============================] - 3s 959us/step - loss: 0.4402 - accuracy: 0.8801
Epoch 92/100
2798/2798 [==============================] - 3s 963us/step - loss: 0.4131 - accuracy: 0.8843
Epoch 93/100
2798/2798 [==============================] - 3s 980us/step - loss: 0.4422 - accuracy: 0.8837
Epoch 94/100
2798/2798 [==============================] - 3s 975us/step - loss: 0.3857 - accuracy: 0.8930
Epoch 95/100
2798/2798 [==============================] - 3s 968us/step - loss: 0.4068 - accuracy: 0.8898
Epoch 96/100
2798/2798 [==============================] - 3s 957us/step - loss: 0.3931 - accuracy: 0.8925
Epoch 97/100
2798/2798 [==============================] - 3s 958us/step - loss: 0.3835 - accuracy: 0.8915
Epoch 98/100
2798/2798 [==============================] - 3s 944us/step - loss: 0.3747 - accuracy: 0.8956
Epoch 99/100
2798/2798 [==============================] - 3s 961us/step - loss: 0.3742 - accuracy: 0.8938
Epoch 100/100
2798/2798 [==============================] - 3s 953us/step - loss: 0.3444 - accuracy: 0.8985

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_4_1_100_epochs", model=model, model_history=history)

Observations

The current neural network has a simple architecture with only two layers, containing 4 and 1 neurons respectively. Despite achieving a steady increase in accuracy, further increasing the number of epochs is unlikely to lead to overfitting. Therefore, to improve performance, it may be necessary to consider expanding the network architecture by adding more layers or increasing the number of neurons.



Model Building with multiple layers and neurons (8-1) and 50 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(8, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=50, verbose=1)
Epoch 1/50
2798/2798 [==============================] - 3s 962us/step - loss: 327.3688 - accuracy: 0.8231
Epoch 2/50
2798/2798 [==============================] - 3s 972us/step - loss: 0.7520 - accuracy: 0.8771
Epoch 3/50
2798/2798 [==============================] - 3s 964us/step - loss: 0.7503 - accuracy: 0.8873
Epoch 4/50
2798/2798 [==============================] - 3s 968us/step - loss: 0.7330 - accuracy: 0.8853
Epoch 5/50
2798/2798 [==============================] - 3s 967us/step - loss: 0.6833 - accuracy: 0.8919
Epoch 6/50
2798/2798 [==============================] - 3s 961us/step - loss: 0.6962 - accuracy: 0.8875
Epoch 7/50
2798/2798 [==============================] - 3s 958us/step - loss: 0.6582 - accuracy: 0.8840
Epoch 8/50
2798/2798 [==============================] - 3s 973us/step - loss: 0.6472 - accuracy: 0.8837
Epoch 9/50
2798/2798 [==============================] - 3s 997us/step - loss: 0.6130 - accuracy: 0.8925
Epoch 10/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6168 - accuracy: 0.8929
Epoch 11/50
2798/2798 [==============================] - 3s 961us/step - loss: 0.6003 - accuracy: 0.8975
Epoch 12/50
2798/2798 [==============================] - 3s 958us/step - loss: 0.4956 - accuracy: 0.9147
Epoch 13/50
2798/2798 [==============================] - 3s 954us/step - loss: 0.4882 - accuracy: 0.9158
Epoch 14/50
2798/2798 [==============================] - 3s 954us/step - loss: 0.4754 - accuracy: 0.9186
Epoch 15/50
2798/2798 [==============================] - 3s 962us/step - loss: 0.4689 - accuracy: 0.9184
Epoch 16/50
2798/2798 [==============================] - 3s 967us/step - loss: 0.4500 - accuracy: 0.9195
Epoch 17/50
2798/2798 [==============================] - 3s 958us/step - loss: 0.4483 - accuracy: 0.9192
Epoch 18/50
2798/2798 [==============================] - 3s 951us/step - loss: 0.4173 - accuracy: 0.9207
Epoch 19/50
2798/2798 [==============================] - 3s 959us/step - loss: 0.4021 - accuracy: 0.9219
Epoch 20/50
2798/2798 [==============================] - 3s 973us/step - loss: 0.3857 - accuracy: 0.9231
Epoch 21/50
2798/2798 [==============================] - 3s 970us/step - loss: 0.3612 - accuracy: 0.9229
Epoch 22/50
2798/2798 [==============================] - 3s 969us/step - loss: 0.3707 - accuracy: 0.9220
Epoch 23/50
2798/2798 [==============================] - 3s 962us/step - loss: 0.3323 - accuracy: 0.9261
Epoch 24/50
2798/2798 [==============================] - 3s 961us/step - loss: 0.3323 - accuracy: 0.9227
Epoch 25/50
2798/2798 [==============================] - 3s 973us/step - loss: 0.3180 - accuracy: 0.9242
Epoch 26/50
2798/2798 [==============================] - 3s 967us/step - loss: 0.3166 - accuracy: 0.9218
Epoch 27/50
2798/2798 [==============================] - 3s 960us/step - loss: 0.2893 - accuracy: 0.9254
Epoch 28/50
2798/2798 [==============================] - 3s 960us/step - loss: 0.2842 - accuracy: 0.9243
Epoch 29/50
2798/2798 [==============================] - 3s 966us/step - loss: 0.2558 - accuracy: 0.9269
Epoch 30/50
2798/2798 [==============================] - 3s 964us/step - loss: 0.2540 - accuracy: 0.9260
Epoch 31/50
2798/2798 [==============================] - 3s 966us/step - loss: 0.2524 - accuracy: 0.9253
Epoch 32/50
2798/2798 [==============================] - 3s 952us/step - loss: 0.2316 - accuracy: 0.9283
Epoch 33/50
2798/2798 [==============================] - 3s 972us/step - loss: 0.2135 - accuracy: 0.9300
Epoch 34/50
2798/2798 [==============================] - 3s 966us/step - loss: 0.2158 - accuracy: 0.9283
Epoch 35/50
2798/2798 [==============================] - 3s 949us/step - loss: 0.1904 - accuracy: 0.9327
Epoch 36/50
2798/2798 [==============================] - 3s 965us/step - loss: 0.1804 - accuracy: 0.9333
Epoch 37/50
2798/2798 [==============================] - 3s 960us/step - loss: 0.1716 - accuracy: 0.9343
Epoch 38/50
2798/2798 [==============================] - 3s 963us/step - loss: 0.1723 - accuracy: 0.9341
Epoch 39/50
2798/2798 [==============================] - 3s 961us/step - loss: 0.1630 - accuracy: 0.9377
Epoch 40/50
2798/2798 [==============================] - 3s 948us/step - loss: 0.1534 - accuracy: 0.9411
Epoch 41/50
2798/2798 [==============================] - 3s 965us/step - loss: 0.1503 - accuracy: 0.9409
Epoch 42/50
2798/2798 [==============================] - 3s 969us/step - loss: 0.1455 - accuracy: 0.9438
Epoch 43/50
2798/2798 [==============================] - 3s 971us/step - loss: 0.1341 - accuracy: 0.9467
Epoch 44/50
2798/2798 [==============================] - 3s 964us/step - loss: 0.1273 - accuracy: 0.9497
Epoch 45/50
2798/2798 [==============================] - 3s 965us/step - loss: 0.1141 - accuracy: 0.9546
Epoch 46/50
2798/2798 [==============================] - 3s 954us/step - loss: 0.1088 - accuracy: 0.9577
Epoch 47/50
2798/2798 [==============================] - 3s 965us/step - loss: 0.1076 - accuracy: 0.9605
Epoch 48/50
2798/2798 [==============================] - 3s 976us/step - loss: 0.1016 - accuracy: 0.9625
Epoch 49/50
2798/2798 [==============================] - 3s 953us/step - loss: 0.0954 - accuracy: 0.9657
Epoch 50/50
2798/2798 [==============================] - 3s 957us/step - loss: 0.0946 - accuracy: 0.9662

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_8_1_50_epochs", model=model, model_history=history)

Observations

The current neural network architecture has two layers with 8 and 1 neurons in each layer, and it is not showing signs of overfitting with 50 epochs. Therefore, it may be worth considering increasing the number of epochs to 100 which might potentially improve the accuracy.



Model Building with multiple layers and neurons (8-1) and 100 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(8, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=100, verbose=1)
Epoch 1/100
2798/2798 [==============================] - 3s 957us/step - loss: 30.9792 - accuracy: 0.7061
Epoch 2/100
2798/2798 [==============================] - 3s 966us/step - loss: 3.1505 - accuracy: 0.8236
Epoch 3/100
2798/2798 [==============================] - 3s 968us/step - loss: 2.7869 - accuracy: 0.8348
Epoch 4/100
2798/2798 [==============================] - 3s 973us/step - loss: 2.5747 - accuracy: 0.8438
Epoch 5/100
2798/2798 [==============================] - 3s 960us/step - loss: 2.4737 - accuracy: 0.8506
Epoch 6/100
2798/2798 [==============================] - 3s 1ms/step - loss: 2.4850 - accuracy: 0.8515
Epoch 7/100
2798/2798 [==============================] - 3s 982us/step - loss: 2.8407 - accuracy: 0.8439
Epoch 8/100
2798/2798 [==============================] - 3s 966us/step - loss: 2.3815 - accuracy: 0.8566
Epoch 9/100
2798/2798 [==============================] - 3s 960us/step - loss: 2.3204 - accuracy: 0.8581
Epoch 10/100
2798/2798 [==============================] - 3s 960us/step - loss: 2.6531 - accuracy: 0.8524
Epoch 11/100
2798/2798 [==============================] - 3s 965us/step - loss: 2.3886 - accuracy: 0.8588
Epoch 12/100
2798/2798 [==============================] - 3s 968us/step - loss: 2.4248 - accuracy: 0.8590
Epoch 13/100
2798/2798 [==============================] - 3s 970us/step - loss: 2.2579 - accuracy: 0.8615
Epoch 14/100
2798/2798 [==============================] - 3s 957us/step - loss: 2.2351 - accuracy: 0.8637
Epoch 15/100
2798/2798 [==============================] - 3s 961us/step - loss: 2.3234 - accuracy: 0.8645
Epoch 16/100
2798/2798 [==============================] - 3s 956us/step - loss: 2.1418 - accuracy: 0.8697
Epoch 17/100
2798/2798 [==============================] - 3s 961us/step - loss: 2.0297 - accuracy: 0.8830
Epoch 18/100
2798/2798 [==============================] - 3s 956us/step - loss: 1.7428 - accuracy: 0.9079
Epoch 19/100
2798/2798 [==============================] - 3s 968us/step - loss: 1.7723 - accuracy: 0.9112
Epoch 20/100
2798/2798 [==============================] - 3s 966us/step - loss: 1.5346 - accuracy: 0.9187
Epoch 21/100
2798/2798 [==============================] - 3s 957us/step - loss: 1.7653 - accuracy: 0.9148
Epoch 22/100
2798/2798 [==============================] - 3s 969us/step - loss: 1.5928 - accuracy: 0.9204
Epoch 23/100
2798/2798 [==============================] - 3s 958us/step - loss: 1.5305 - accuracy: 0.9243
Epoch 24/100
2798/2798 [==============================] - 3s 955us/step - loss: 1.4368 - accuracy: 0.9290
Epoch 25/100
2798/2798 [==============================] - 3s 993us/step - loss: 1.4672 - accuracy: 0.9311
Epoch 26/100
2798/2798 [==============================] - 3s 1ms/step - loss: 1.3299 - accuracy: 0.9341
Epoch 27/100
2798/2798 [==============================] - 3s 956us/step - loss: 1.2504 - accuracy: 0.9381
Epoch 28/100
2798/2798 [==============================] - 3s 971us/step - loss: 1.3634 - accuracy: 0.9343
Epoch 29/100
2798/2798 [==============================] - 3s 962us/step - loss: 1.3071 - accuracy: 0.9371
Epoch 30/100
2798/2798 [==============================] - 3s 951us/step - loss: 1.3470 - accuracy: 0.9376
Epoch 31/100
2798/2798 [==============================] - 3s 956us/step - loss: 1.2608 - accuracy: 0.9396
Epoch 32/100
2798/2798 [==============================] - 3s 963us/step - loss: 1.3270 - accuracy: 0.9374
Epoch 33/100
2798/2798 [==============================] - 3s 965us/step - loss: 1.3230 - accuracy: 0.9390
Epoch 34/100
2798/2798 [==============================] - 3s 968us/step - loss: 1.2317 - accuracy: 0.9417
Epoch 35/100
2798/2798 [==============================] - 3s 950us/step - loss: 1.2336 - accuracy: 0.9399
Epoch 36/100
2798/2798 [==============================] - 3s 955us/step - loss: 1.3293 - accuracy: 0.9395
Epoch 37/100
2798/2798 [==============================] - 3s 959us/step - loss: 1.3927 - accuracy: 0.9388
Epoch 38/100
2798/2798 [==============================] - 3s 958us/step - loss: 1.2014 - accuracy: 0.9443
Epoch 39/100
2798/2798 [==============================] - 3s 954us/step - loss: 1.3341 - accuracy: 0.9410
Epoch 40/100
2798/2798 [==============================] - 3s 963us/step - loss: 1.4120 - accuracy: 0.9417
Epoch 41/100
2798/2798 [==============================] - 3s 950us/step - loss: 1.2920 - accuracy: 0.9423
Epoch 42/100
2798/2798 [==============================] - 3s 965us/step - loss: 1.2605 - accuracy: 0.9436
Epoch 43/100
2798/2798 [==============================] - 3s 955us/step - loss: 1.1782 - accuracy: 0.9446
Epoch 44/100
2798/2798 [==============================] - 3s 956us/step - loss: 1.2640 - accuracy: 0.9424
Epoch 45/100
2798/2798 [==============================] - 3s 967us/step - loss: 1.3521 - accuracy: 0.9439
Epoch 46/100
2798/2798 [==============================] - 3s 967us/step - loss: 1.2422 - accuracy: 0.9461
Epoch 47/100
2798/2798 [==============================] - 3s 958us/step - loss: 1.3161 - accuracy: 0.9448
Epoch 48/100
2798/2798 [==============================] - 3s 958us/step - loss: 1.1908 - accuracy: 0.9499
Epoch 49/100
2798/2798 [==============================] - 3s 945us/step - loss: 1.2612 - accuracy: 0.9512
Epoch 50/100
2798/2798 [==============================] - 3s 962us/step - loss: 1.2412 - accuracy: 0.9524
Epoch 51/100
2798/2798 [==============================] - 3s 966us/step - loss: 1.2530 - accuracy: 0.9501
Epoch 52/100
2798/2798 [==============================] - 3s 965us/step - loss: 1.2275 - accuracy: 0.9527
Epoch 53/100
2798/2798 [==============================] - 3s 950us/step - loss: 1.2890 - accuracy: 0.9519
Epoch 54/100
2798/2798 [==============================] - 3s 956us/step - loss: 1.2407 - accuracy: 0.9517
Epoch 55/100
2798/2798 [==============================] - 3s 959us/step - loss: 1.2179 - accuracy: 0.9542
Epoch 56/100
2798/2798 [==============================] - 3s 961us/step - loss: 1.2181 - accuracy: 0.9542
Epoch 57/100
2798/2798 [==============================] - 3s 967us/step - loss: 1.1606 - accuracy: 0.9548
Epoch 58/100
2798/2798 [==============================] - 3s 960us/step - loss: 1.0381 - accuracy: 0.9593
Epoch 59/100
2798/2798 [==============================] - 3s 950us/step - loss: 1.1793 - accuracy: 0.9558
Epoch 60/100
2798/2798 [==============================] - 3s 959us/step - loss: 1.2135 - accuracy: 0.9522
Epoch 61/100
2798/2798 [==============================] - 3s 973us/step - loss: 1.0947 - accuracy: 0.9573
Epoch 62/100
2798/2798 [==============================] - 3s 971us/step - loss: 1.1077 - accuracy: 0.9562
Epoch 63/100
2798/2798 [==============================] - 3s 965us/step - loss: 1.1289 - accuracy: 0.9562
Epoch 64/100
2798/2798 [==============================] - 3s 965us/step - loss: 1.1737 - accuracy: 0.9552
Epoch 65/100
2798/2798 [==============================] - 3s 972us/step - loss: 1.1834 - accuracy: 0.9549
Epoch 66/100
2798/2798 [==============================] - 3s 959us/step - loss: 1.0539 - accuracy: 0.9580
Epoch 67/100
2798/2798 [==============================] - 3s 975us/step - loss: 1.1098 - accuracy: 0.9560
Epoch 68/100
2798/2798 [==============================] - 3s 970us/step - loss: 1.0561 - accuracy: 0.9576
Epoch 69/100
2798/2798 [==============================] - 3s 958us/step - loss: 1.0852 - accuracy: 0.9564
Epoch 70/100
2798/2798 [==============================] - 3s 959us/step - loss: 1.0899 - accuracy: 0.9562
Epoch 71/100
2798/2798 [==============================] - 3s 956us/step - loss: 0.9956 - accuracy: 0.9580
Epoch 72/100
2798/2798 [==============================] - 3s 965us/step - loss: 0.9995 - accuracy: 0.9597
Epoch 73/100
2798/2798 [==============================] - 3s 958us/step - loss: 1.0001 - accuracy: 0.9576
Epoch 74/100
2798/2798 [==============================] - 3s 961us/step - loss: 0.9437 - accuracy: 0.9610
Epoch 75/100
2798/2798 [==============================] - 3s 946us/step - loss: 0.9881 - accuracy: 0.9582
Epoch 76/100
2798/2798 [==============================] - 3s 960us/step - loss: 0.9571 - accuracy: 0.9594
Epoch 77/100
2798/2798 [==============================] - 3s 954us/step - loss: 0.9811 - accuracy: 0.9582
Epoch 78/100
2798/2798 [==============================] - 3s 955us/step - loss: 0.9611 - accuracy: 0.9569
Epoch 79/100
2798/2798 [==============================] - 3s 957us/step - loss: 0.9142 - accuracy: 0.9603
Epoch 80/100
2798/2798 [==============================] - 3s 952us/step - loss: 0.9280 - accuracy: 0.9597
Epoch 81/100
2798/2798 [==============================] - 3s 956us/step - loss: 0.9390 - accuracy: 0.9591
Epoch 82/100
2798/2798 [==============================] - 3s 966us/step - loss: 0.8358 - accuracy: 0.9628
Epoch 83/100
2798/2798 [==============================] - 3s 956us/step - loss: 0.9470 - accuracy: 0.9584
Epoch 84/100
2798/2798 [==============================] - 3s 953us/step - loss: 0.8985 - accuracy: 0.9597
Epoch 85/100
2798/2798 [==============================] - 3s 943us/step - loss: 0.8449 - accuracy: 0.9622
Epoch 86/100
2798/2798 [==============================] - 3s 948us/step - loss: 0.9236 - accuracy: 0.9579
Epoch 87/100
2798/2798 [==============================] - 3s 960us/step - loss: 0.8474 - accuracy: 0.9624
Epoch 88/100
2798/2798 [==============================] - 3s 966us/step - loss: 0.8991 - accuracy: 0.9597
Epoch 89/100
2798/2798 [==============================] - 3s 961us/step - loss: 0.7824 - accuracy: 0.9627
Epoch 90/100
2798/2798 [==============================] - 3s 954us/step - loss: 0.8896 - accuracy: 0.9592
Epoch 91/100
2798/2798 [==============================] - 3s 970us/step - loss: 0.8662 - accuracy: 0.9595
Epoch 92/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8220 - accuracy: 0.9620
Epoch 93/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8157 - accuracy: 0.9635
Epoch 94/100
2798/2798 [==============================] - 3s 966us/step - loss: 0.8047 - accuracy: 0.9611
Epoch 95/100
2798/2798 [==============================] - 3s 961us/step - loss: 0.8133 - accuracy: 0.9611
Epoch 96/100
2798/2798 [==============================] - 3s 971us/step - loss: 0.7977 - accuracy: 0.9596
Epoch 97/100
2798/2798 [==============================] - 3s 970us/step - loss: 0.8067 - accuracy: 0.9602
Epoch 98/100
2798/2798 [==============================] - 3s 961us/step - loss: 0.7771 - accuracy: 0.9612
Epoch 99/100
2798/2798 [==============================] - 3s 962us/step - loss: 0.7683 - accuracy: 0.9616
Epoch 100/100
2798/2798 [==============================] - 3s 958us/step - loss: 0.7741 - accuracy: 0.9616

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_8_1_100_epochs", model=model, model_history=history)

Observations

Based on the fact that the current neural network architecture, which consists of two layers with 8 and 1 neurons, is not achieving 100 percent convergence even after 100 epochs, it might be worth exploring the option of increasing the size of the neural network by adding more layers and neurons.



Model Building with multiple layers and neurons (4-4-1) and 50 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(4, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(4, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=50, verbose=1)
Epoch 1/50
2798/2798 [==============================] - 4s 1ms/step - loss: 21.3935 - accuracy: 0.3716
Epoch 2/50
2798/2798 [==============================] - 3s 1ms/step - loss: 1.6386 - accuracy: 0.6124
Epoch 3/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6509 - accuracy: 0.7891
Epoch 4/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5990 - accuracy: 0.8184
Epoch 5/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6168 - accuracy: 0.8255
Epoch 6/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5504 - accuracy: 0.8410
Epoch 7/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5436 - accuracy: 0.8404
Epoch 8/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5676 - accuracy: 0.8370
Epoch 9/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5173 - accuracy: 0.8482
Epoch 10/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4983 - accuracy: 0.8519
Epoch 11/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5204 - accuracy: 0.8471
Epoch 12/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4938 - accuracy: 0.8535
Epoch 13/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4944 - accuracy: 0.8511
Epoch 14/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4798 - accuracy: 0.8558
Epoch 15/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4660 - accuracy: 0.8571
Epoch 16/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4593 - accuracy: 0.8597
Epoch 17/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4724 - accuracy: 0.8553
Epoch 18/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4581 - accuracy: 0.8601
Epoch 19/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4677 - accuracy: 0.8572
Epoch 20/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4536 - accuracy: 0.8596
Epoch 21/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4539 - accuracy: 0.8585
Epoch 22/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4584 - accuracy: 0.8608
Epoch 23/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4515 - accuracy: 0.8600
Epoch 24/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4365 - accuracy: 0.8645
Epoch 25/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4313 - accuracy: 0.8644
Epoch 26/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4030 - accuracy: 0.8692
Epoch 27/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4342 - accuracy: 0.8623
Epoch 28/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4372 - accuracy: 0.8594
Epoch 29/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4072 - accuracy: 0.8663
Epoch 30/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4225 - accuracy: 0.8641
Epoch 31/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4146 - accuracy: 0.8665
Epoch 32/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4139 - accuracy: 0.8628
Epoch 33/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3874 - accuracy: 0.8666
Epoch 34/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3810 - accuracy: 0.8676
Epoch 35/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3879 - accuracy: 0.8677
Epoch 36/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3684 - accuracy: 0.8717
Epoch 37/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3769 - accuracy: 0.8682
Epoch 38/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3783 - accuracy: 0.8655
Epoch 39/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3603 - accuracy: 0.8715
Epoch 40/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3618 - accuracy: 0.8699
Epoch 41/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3485 - accuracy: 0.8701
Epoch 42/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3438 - accuracy: 0.8731
Epoch 43/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3450 - accuracy: 0.8719
Epoch 44/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3443 - accuracy: 0.8721
Epoch 45/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3396 - accuracy: 0.8712
Epoch 46/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3252 - accuracy: 0.8755
Epoch 47/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3373 - accuracy: 0.8698
Epoch 48/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3230 - accuracy: 0.8764
Epoch 49/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3251 - accuracy: 0.8718
Epoch 50/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3214 - accuracy: 0.8731

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_4_4_1_50_epochs", model=model, model_history=history)

Observations

The neural network consists of three layers, each having 4, 4, and 1 neurons. The model has been trained for 50 rounds and achieved an accuracy of 0.8731, without exhibiting any issues related to overfitting or convergence. It is possible that increasing the number of epochs could result in overfitting.



Model Building with multiple layers and neurons (4-4-1) and 100 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(4, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(4, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=100, verbose=1)
Epoch 1/100
2798/2798 [==============================] - 4s 1ms/step - loss: 1.6053 - accuracy: 0.7265
Epoch 2/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5742 - accuracy: 0.8089
Epoch 3/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5149 - accuracy: 0.8273
Epoch 4/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5549 - accuracy: 0.8304
Epoch 5/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4761 - accuracy: 0.8474
Epoch 6/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4137 - accuracy: 0.8614
Epoch 7/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3470 - accuracy: 0.9042
Epoch 8/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3773 - accuracy: 0.9054
Epoch 9/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3129 - accuracy: 0.9139
Epoch 10/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3000 - accuracy: 0.9213
Epoch 11/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3021 - accuracy: 0.9232
Epoch 12/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2819 - accuracy: 0.9275
Epoch 13/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2557 - accuracy: 0.9349
Epoch 14/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2079 - accuracy: 0.9460
Epoch 15/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1835 - accuracy: 0.9508
Epoch 16/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1726 - accuracy: 0.9535
Epoch 17/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1606 - accuracy: 0.9558
Epoch 18/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1372 - accuracy: 0.9616
Epoch 19/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1489 - accuracy: 0.9558
Epoch 20/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1123 - accuracy: 0.9628
Epoch 21/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1130 - accuracy: 0.9618
Epoch 22/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1120 - accuracy: 0.9621
Epoch 23/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0968 - accuracy: 0.9658
Epoch 24/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1237 - accuracy: 0.9632
Epoch 25/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1079 - accuracy: 0.9583
Epoch 26/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1075 - accuracy: 0.9582
Epoch 27/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1017 - accuracy: 0.9610
Epoch 28/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0996 - accuracy: 0.9615
Epoch 29/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1215 - accuracy: 0.9601
Epoch 30/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1049 - accuracy: 0.9611
Epoch 31/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1023 - accuracy: 0.9610
Epoch 32/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0982 - accuracy: 0.9626
Epoch 33/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1034 - accuracy: 0.9630
Epoch 34/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0897 - accuracy: 0.9692
Epoch 35/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0968 - accuracy: 0.9671
Epoch 36/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1026 - accuracy: 0.9607
Epoch 37/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0955 - accuracy: 0.9636
Epoch 38/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0967 - accuracy: 0.9656
Epoch 39/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0903 - accuracy: 0.9662
Epoch 40/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0940 - accuracy: 0.9668
Epoch 41/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1388 - accuracy: 0.9694
Epoch 42/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0871 - accuracy: 0.9703
Epoch 43/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0977 - accuracy: 0.9699
Epoch 44/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0940 - accuracy: 0.9675
Epoch 45/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0880 - accuracy: 0.9690
Epoch 46/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0819 - accuracy: 0.9720
Epoch 47/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0898 - accuracy: 0.9701
Epoch 48/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0971 - accuracy: 0.9636
Epoch 49/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0861 - accuracy: 0.9716
Epoch 50/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1306 - accuracy: 0.9722
Epoch 51/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0830 - accuracy: 0.9723
Epoch 52/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0926 - accuracy: 0.9700
Epoch 53/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0901 - accuracy: 0.9731
Epoch 54/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0872 - accuracy: 0.9743
Epoch 55/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0790 - accuracy: 0.9747
Epoch 56/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0924 - accuracy: 0.9744
Epoch 57/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0754 - accuracy: 0.9763
Epoch 58/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0739 - accuracy: 0.9767
Epoch 59/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0706 - accuracy: 0.9784
Epoch 60/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0735 - accuracy: 0.9769
Epoch 61/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0725 - accuracy: 0.9778
Epoch 62/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0757 - accuracy: 0.9770
Epoch 63/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0705 - accuracy: 0.9773
Epoch 64/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0703 - accuracy: 0.9792
Epoch 65/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0764 - accuracy: 0.9767
Epoch 66/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0652 - accuracy: 0.9801
Epoch 67/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0693 - accuracy: 0.9794
Epoch 68/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3764 - accuracy: 0.8454
Epoch 69/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6750 - accuracy: 0.5739
Epoch 70/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6391 - accuracy: 0.5741
Epoch 71/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6421 - accuracy: 0.5705
Epoch 72/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6406 - accuracy: 0.5732
Epoch 73/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6374 - accuracy: 0.5765
Epoch 74/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6420 - accuracy: 0.5709
Epoch 75/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6374 - accuracy: 0.5766
Epoch 76/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6420 - accuracy: 0.5785
Epoch 77/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6387 - accuracy: 0.5748
Epoch 78/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6364 - accuracy: 0.5776
Epoch 79/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6389 - accuracy: 0.5743
Epoch 80/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6388 - accuracy: 0.5747
Epoch 81/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6386 - accuracy: 0.5747
Epoch 82/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6395 - accuracy: 0.5739
Epoch 83/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6359 - accuracy: 0.5781
Epoch 84/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6394 - accuracy: 0.5739
Epoch 85/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6391 - accuracy: 0.5740
Epoch 86/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6387 - accuracy: 0.5750
Epoch 87/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6405 - accuracy: 0.5726
Epoch 88/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6376 - accuracy: 0.5761
Epoch 89/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6363 - accuracy: 0.5776
Epoch 90/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6363 - accuracy: 0.5780
Epoch 91/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6360 - accuracy: 0.5779
Epoch 92/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6358 - accuracy: 0.5782
Epoch 93/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6354 - accuracy: 0.5787
Epoch 94/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6360 - accuracy: 0.5782
Epoch 95/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6355 - accuracy: 0.5785
Epoch 96/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6389 - accuracy: 0.5751
Epoch 97/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6369 - accuracy: 0.5769
Epoch 98/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6360 - accuracy: 0.5780
Epoch 99/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6373 - accuracy: 0.5763
Epoch 100/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6379 - accuracy: 0.5756

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_4_4_1_100_epochs", model=model, model_history=history)

Observations

The current neural network has three layers with 4, 4, and 1 neurons in each layer. It was trained for 100 epochs and achieved an accuracy of 0.9789, but accuracy is getting decreased after 50 epochs, overfitting was observed. Hence, it might be best to use 50 epochs as the optimal number for training the model. To improve the model's accuracy, we can increase the number of neurons and layers by modifying the model architecture.



Model Building with multiple layers and neurons (4-8-1) and 50 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(4, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(8, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=50, verbose=1)
Epoch 1/50
2798/2798 [==============================] - 4s 1ms/step - loss: 1.9856 - accuracy: 0.5533
Epoch 2/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4305 - accuracy: 0.8305
Epoch 3/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2604 - accuracy: 0.8978
Epoch 4/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2301 - accuracy: 0.9116
Epoch 5/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2174 - accuracy: 0.9189
Epoch 6/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2056 - accuracy: 0.9236
Epoch 7/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1906 - accuracy: 0.9296
Epoch 8/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1772 - accuracy: 0.9367
Epoch 9/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1705 - accuracy: 0.9398
Epoch 10/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1584 - accuracy: 0.9478
Epoch 11/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1498 - accuracy: 0.9518
Epoch 12/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1414 - accuracy: 0.9547
Epoch 13/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1357 - accuracy: 0.9556
Epoch 14/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1228 - accuracy: 0.9602
Epoch 15/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1112 - accuracy: 0.9642
Epoch 16/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1066 - accuracy: 0.9663
Epoch 17/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1016 - accuracy: 0.9680
Epoch 18/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0982 - accuracy: 0.9693
Epoch 19/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1011 - accuracy: 0.9686
Epoch 20/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0941 - accuracy: 0.9704
Epoch 21/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0961 - accuracy: 0.9705
Epoch 22/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0950 - accuracy: 0.9708
Epoch 23/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0942 - accuracy: 0.9711
Epoch 24/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0942 - accuracy: 0.9714
Epoch 25/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0930 - accuracy: 0.9714
Epoch 26/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0928 - accuracy: 0.9718
Epoch 27/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0912 - accuracy: 0.9722
Epoch 28/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0929 - accuracy: 0.9713
Epoch 29/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0934 - accuracy: 0.9722
Epoch 30/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0915 - accuracy: 0.9724
Epoch 31/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0917 - accuracy: 0.9721
Epoch 32/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0907 - accuracy: 0.9726
Epoch 33/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0904 - accuracy: 0.9726
Epoch 34/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0899 - accuracy: 0.9729
Epoch 35/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0909 - accuracy: 0.9726
Epoch 36/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0894 - accuracy: 0.9730
Epoch 37/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0897 - accuracy: 0.9733
Epoch 38/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0910 - accuracy: 0.9731
Epoch 39/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0910 - accuracy: 0.9731
Epoch 40/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0909 - accuracy: 0.9727
Epoch 41/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0908 - accuracy: 0.9730
Epoch 42/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0901 - accuracy: 0.9731
Epoch 43/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0894 - accuracy: 0.9729
Epoch 44/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0892 - accuracy: 0.9732
Epoch 45/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0897 - accuracy: 0.9733
Epoch 46/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0875 - accuracy: 0.9737
Epoch 47/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0874 - accuracy: 0.9741
Epoch 48/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0883 - accuracy: 0.9736
Epoch 49/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0901 - accuracy: 0.9732
Epoch 50/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0882 - accuracy: 0.9736

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_4_8_1_50_epochs", model=model, model_history=history)

Observations

The neural network consists of three layers, each having 4, 8, and 1 neurons. The model has been trained for 50 rounds and achieved an accuracy of 0.9736, without exhibiting any issues related to overfitting or convergence. It is possible that increasing the number of epochs could result in overfitting.



Model Building with multiple layers and neurons (4-8-1) and 100 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(4, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(8, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=100, verbose=1)
Epoch 1/100
2798/2798 [==============================] - 4s 1ms/step - loss: 14.4210 - accuracy: 0.5008
Epoch 2/100
2798/2798 [==============================] - 3s 1ms/step - loss: 2.8302 - accuracy: 0.5500
Epoch 3/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.9552 - accuracy: 0.7730
Epoch 4/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.9641 - accuracy: 0.8465
Epoch 5/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8786 - accuracy: 0.8720
Epoch 6/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8257 - accuracy: 0.8807
Epoch 7/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8117 - accuracy: 0.8856
Epoch 8/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8568 - accuracy: 0.8864
Epoch 9/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8696 - accuracy: 0.8911
Epoch 10/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8149 - accuracy: 0.8955
Epoch 11/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.7604 - accuracy: 0.8967
Epoch 12/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.7823 - accuracy: 0.8991
Epoch 13/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8020 - accuracy: 0.8966
Epoch 14/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.7167 - accuracy: 0.9041
Epoch 15/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.7032 - accuracy: 0.9005
Epoch 16/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.7348 - accuracy: 0.8991
Epoch 17/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6694 - accuracy: 0.9013
Epoch 18/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6680 - accuracy: 0.9016
Epoch 19/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6000 - accuracy: 0.9056
Epoch 20/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5300 - accuracy: 0.9121
Epoch 21/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6421 - accuracy: 0.9062
Epoch 22/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5449 - accuracy: 0.9188
Epoch 23/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5612 - accuracy: 0.9147
Epoch 24/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5170 - accuracy: 0.9196
Epoch 25/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5655 - accuracy: 0.9148
Epoch 26/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4945 - accuracy: 0.9221
Epoch 27/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4503 - accuracy: 0.9212
Epoch 28/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4679 - accuracy: 0.9206
Epoch 29/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4197 - accuracy: 0.9237
Epoch 30/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4819 - accuracy: 0.9201
Epoch 31/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3979 - accuracy: 0.9278
Epoch 32/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3879 - accuracy: 0.9276
Epoch 33/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3925 - accuracy: 0.9227
Epoch 34/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3600 - accuracy: 0.9289
Epoch 35/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3298 - accuracy: 0.9293
Epoch 36/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2903 - accuracy: 0.9292
Epoch 37/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2809 - accuracy: 0.9326
Epoch 38/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2782 - accuracy: 0.9304
Epoch 39/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2481 - accuracy: 0.9318
Epoch 40/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2560 - accuracy: 0.9323
Epoch 41/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2480 - accuracy: 0.9333
Epoch 42/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2090 - accuracy: 0.9388
Epoch 43/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2299 - accuracy: 0.9357
Epoch 44/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2055 - accuracy: 0.9338
Epoch 45/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2234 - accuracy: 0.9362
Epoch 46/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1932 - accuracy: 0.9369
Epoch 47/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1843 - accuracy: 0.9379
Epoch 48/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2227 - accuracy: 0.9353
Epoch 49/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1910 - accuracy: 0.9364
Epoch 50/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2080 - accuracy: 0.9376
Epoch 51/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1822 - accuracy: 0.9376
Epoch 52/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1953 - accuracy: 0.9368
Epoch 53/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1954 - accuracy: 0.9384
Epoch 54/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2054 - accuracy: 0.9386
Epoch 55/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1860 - accuracy: 0.9394
Epoch 56/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2014 - accuracy: 0.9410
Epoch 57/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1799 - accuracy: 0.9398
Epoch 58/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1937 - accuracy: 0.9376
Epoch 59/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1930 - accuracy: 0.9409
Epoch 60/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1810 - accuracy: 0.9418
Epoch 61/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1765 - accuracy: 0.9405
Epoch 62/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2139 - accuracy: 0.9417
Epoch 63/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1991 - accuracy: 0.9408
Epoch 64/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1756 - accuracy: 0.9437
Epoch 65/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1795 - accuracy: 0.9419
Epoch 66/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1819 - accuracy: 0.9430
Epoch 67/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1993 - accuracy: 0.9432
Epoch 68/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1838 - accuracy: 0.9436
Epoch 69/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1697 - accuracy: 0.9437
Epoch 70/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1654 - accuracy: 0.9456
Epoch 71/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1804 - accuracy: 0.9437
Epoch 72/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1782 - accuracy: 0.9452
Epoch 73/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1824 - accuracy: 0.9467
Epoch 74/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1685 - accuracy: 0.9462
Epoch 75/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1837 - accuracy: 0.9470
Epoch 76/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1725 - accuracy: 0.9477
Epoch 77/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1593 - accuracy: 0.9477
Epoch 78/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1903 - accuracy: 0.9460
Epoch 79/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1872 - accuracy: 0.9467
Epoch 80/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1390 - accuracy: 0.9495
Epoch 81/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1764 - accuracy: 0.9485
Epoch 82/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1667 - accuracy: 0.9494
Epoch 83/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2099 - accuracy: 0.9491
Epoch 84/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1760 - accuracy: 0.9478
Epoch 85/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1875 - accuracy: 0.9499
Epoch 86/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1403 - accuracy: 0.9502
Epoch 87/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1433 - accuracy: 0.9500
Epoch 88/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1506 - accuracy: 0.9494
Epoch 89/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1549 - accuracy: 0.9493
Epoch 90/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1364 - accuracy: 0.9530
Epoch 91/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1412 - accuracy: 0.9500
Epoch 92/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1603 - accuracy: 0.9518
Epoch 93/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1342 - accuracy: 0.9521
Epoch 94/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1287 - accuracy: 0.9531
Epoch 95/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1517 - accuracy: 0.9509
Epoch 96/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1293 - accuracy: 0.9531
Epoch 97/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1313 - accuracy: 0.9527
Epoch 98/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1567 - accuracy: 0.9527
Epoch 99/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1411 - accuracy: 0.9539
Epoch 100/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1452 - accuracy: 0.9526

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_4_8_1_100_epochs", model=model, model_history=history)

Observations

The current neural network has three layers with 4, 8, and 1 neurons in each layer. It was trained for 100 epochs and achieved an accuracy of 0.9526, but overfitting was observed. Hence, it might be best to use 100 epochs as the optimal number for training the model. To improve the model's accuracy, we can increase the number of neurons and layers by modifying the model architecture.



Model Building with multiple layers and neurons (8-4-1) and 50 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(8, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(4, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=50, verbose=1)
Epoch 1/50
2798/2798 [==============================] - 4s 1ms/step - loss: 30.5770 - accuracy: 0.4190
Epoch 2/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.7043 - accuracy: 0.6263
Epoch 3/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3660 - accuracy: 0.8542
Epoch 4/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2965 - accuracy: 0.8867
Epoch 5/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2581 - accuracy: 0.9046
Epoch 6/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2312 - accuracy: 0.9182
Epoch 7/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2149 - accuracy: 0.9244
Epoch 8/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2061 - accuracy: 0.9287
Epoch 9/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1948 - accuracy: 0.9314
Epoch 10/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1786 - accuracy: 0.9361
Epoch 11/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1637 - accuracy: 0.9420
Epoch 12/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1497 - accuracy: 0.9463
Epoch 13/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1364 - accuracy: 0.9524
Epoch 14/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1269 - accuracy: 0.9569
Epoch 15/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1203 - accuracy: 0.9584
Epoch 16/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1111 - accuracy: 0.9617
Epoch 17/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1105 - accuracy: 0.9629
Epoch 18/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1092 - accuracy: 0.9631
Epoch 19/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1091 - accuracy: 0.9655
Epoch 20/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1040 - accuracy: 0.9665
Epoch 21/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1019 - accuracy: 0.9664
Epoch 22/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1005 - accuracy: 0.9670
Epoch 23/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0962 - accuracy: 0.9683
Epoch 24/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0991 - accuracy: 0.9676
Epoch 25/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1013 - accuracy: 0.9687
Epoch 26/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0961 - accuracy: 0.9687
Epoch 27/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0964 - accuracy: 0.9691
Epoch 28/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0937 - accuracy: 0.9696
Epoch 29/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0917 - accuracy: 0.9704
Epoch 30/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0943 - accuracy: 0.9694
Epoch 31/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0932 - accuracy: 0.9700
Epoch 32/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0925 - accuracy: 0.9706
Epoch 33/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0912 - accuracy: 0.9717
Epoch 34/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0905 - accuracy: 0.9717
Epoch 35/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0900 - accuracy: 0.9716
Epoch 36/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0895 - accuracy: 0.9720
Epoch 37/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0894 - accuracy: 0.9715
Epoch 38/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0874 - accuracy: 0.9732
Epoch 39/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0883 - accuracy: 0.9720
Epoch 40/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0871 - accuracy: 0.9728
Epoch 41/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0870 - accuracy: 0.9729
Epoch 42/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0870 - accuracy: 0.9726
Epoch 43/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0879 - accuracy: 0.9719
Epoch 44/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0876 - accuracy: 0.9731
Epoch 45/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0854 - accuracy: 0.9729
Epoch 46/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0854 - accuracy: 0.9736
Epoch 47/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0860 - accuracy: 0.9740
Epoch 48/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0836 - accuracy: 0.9746
Epoch 49/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0859 - accuracy: 0.9735
Epoch 50/50
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0833 - accuracy: 0.9743

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_8_4_1_50_epochs", model=model, model_history=history)

Observations

The current neural network has three layers with 8, 4, and 1 neurons in each layer. After being trained for 50 epochs, the model achieved an accuracy of 0.9743, but it showed signs of overfitting or convergence problems slowly. As a result, it may be beneficial to increse the number of epochs for training the model.



Model Building with multiple layers and neurons (8-4-1) and 100 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(8, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(4, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
model.fit(x=X_smote, y=y_smote, epochs=100, verbose=1)
Epoch 1/100
2798/2798 [==============================] - 4s 1ms/step - loss: 11.7179 - accuracy: 0.6679
Epoch 2/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3442 - accuracy: 0.8311
Epoch 3/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2632 - accuracy: 0.9136
Epoch 4/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2269 - accuracy: 0.9287
Epoch 5/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2076 - accuracy: 0.9347
Epoch 6/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1855 - accuracy: 0.9395
Epoch 7/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1693 - accuracy: 0.9444
Epoch 8/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1565 - accuracy: 0.9479
Epoch 9/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1438 - accuracy: 0.9512
Epoch 10/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1396 - accuracy: 0.9522
Epoch 11/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1334 - accuracy: 0.9547
Epoch 12/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1282 - accuracy: 0.9570
Epoch 13/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1260 - accuracy: 0.9574
Epoch 14/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1203 - accuracy: 0.9597
Epoch 15/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1185 - accuracy: 0.9613
Epoch 16/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1164 - accuracy: 0.9619
Epoch 17/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1163 - accuracy: 0.9620
Epoch 18/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1146 - accuracy: 0.9620
Epoch 19/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1125 - accuracy: 0.9628
Epoch 20/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1111 - accuracy: 0.9634
Epoch 21/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1071 - accuracy: 0.9650
Epoch 22/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1080 - accuracy: 0.9651
Epoch 23/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1051 - accuracy: 0.9659
Epoch 24/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1050 - accuracy: 0.9664
Epoch 25/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1046 - accuracy: 0.9660
Epoch 26/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1036 - accuracy: 0.9670
Epoch 27/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1025 - accuracy: 0.9671
Epoch 28/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1014 - accuracy: 0.9682
Epoch 29/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1002 - accuracy: 0.9682
Epoch 30/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1015 - accuracy: 0.9674
Epoch 31/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0991 - accuracy: 0.9686
Epoch 32/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0999 - accuracy: 0.9682
Epoch 33/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0983 - accuracy: 0.9687
Epoch 34/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0980 - accuracy: 0.9689
Epoch 35/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0967 - accuracy: 0.9702
Epoch 36/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0959 - accuracy: 0.9694
Epoch 37/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0971 - accuracy: 0.9695
Epoch 38/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0953 - accuracy: 0.9703
Epoch 39/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0959 - accuracy: 0.9699
Epoch 40/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0963 - accuracy: 0.9697
Epoch 41/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0939 - accuracy: 0.9704
Epoch 42/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0940 - accuracy: 0.9701
Epoch 43/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0926 - accuracy: 0.9709
Epoch 44/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0929 - accuracy: 0.9704
Epoch 45/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0912 - accuracy: 0.9715
Epoch 46/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0920 - accuracy: 0.9709
Epoch 47/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0914 - accuracy: 0.9710
Epoch 48/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0906 - accuracy: 0.9719
Epoch 49/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0904 - accuracy: 0.9716
Epoch 50/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0909 - accuracy: 0.9720
Epoch 51/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0903 - accuracy: 0.9717
Epoch 52/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0895 - accuracy: 0.9719
Epoch 53/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0900 - accuracy: 0.9725
Epoch 54/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0904 - accuracy: 0.9717
Epoch 55/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0905 - accuracy: 0.9719
Epoch 56/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0890 - accuracy: 0.9724
Epoch 57/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0889 - accuracy: 0.9725
Epoch 58/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0887 - accuracy: 0.9728
Epoch 59/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0888 - accuracy: 0.9726
Epoch 60/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0879 - accuracy: 0.9727
Epoch 61/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0888 - accuracy: 0.9733
Epoch 62/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0852 - accuracy: 0.9742
Epoch 63/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0846 - accuracy: 0.9747
Epoch 64/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0836 - accuracy: 0.9747
Epoch 65/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0836 - accuracy: 0.9748
Epoch 66/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0826 - accuracy: 0.9753
Epoch 67/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0827 - accuracy: 0.9744
Epoch 68/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0829 - accuracy: 0.9749
Epoch 69/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0812 - accuracy: 0.9751
Epoch 70/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0817 - accuracy: 0.9751
Epoch 71/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0803 - accuracy: 0.9754
Epoch 72/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0812 - accuracy: 0.9754
Epoch 73/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0796 - accuracy: 0.9754
Epoch 74/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0796 - accuracy: 0.9757
Epoch 75/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0807 - accuracy: 0.9750
Epoch 76/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0782 - accuracy: 0.9758
Epoch 77/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0783 - accuracy: 0.9760
Epoch 78/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0746 - accuracy: 0.9771
Epoch 79/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0724 - accuracy: 0.9773
Epoch 80/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0711 - accuracy: 0.9782
Epoch 81/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0702 - accuracy: 0.9778
Epoch 82/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0709 - accuracy: 0.9781
Epoch 83/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0675 - accuracy: 0.9788
Epoch 84/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0683 - accuracy: 0.9780
Epoch 85/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0654 - accuracy: 0.9789
Epoch 86/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0645 - accuracy: 0.9791
Epoch 87/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0629 - accuracy: 0.9796
Epoch 88/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0612 - accuracy: 0.9803
Epoch 89/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0597 - accuracy: 0.9802
Epoch 90/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0619 - accuracy: 0.9800
Epoch 91/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0609 - accuracy: 0.9802
Epoch 92/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0591 - accuracy: 0.9807
Epoch 93/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0593 - accuracy: 0.9809
Epoch 94/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0566 - accuracy: 0.9811
Epoch 95/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0614 - accuracy: 0.9806
Epoch 96/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0579 - accuracy: 0.9812
Epoch 97/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0571 - accuracy: 0.9817
Epoch 98/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0566 - accuracy: 0.9816
Epoch 99/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0579 - accuracy: 0.9813
Epoch 100/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0551 - accuracy: 0.9817

<keras.callbacks.History at 0x7fbf7bbc1d20>
# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_8_4_1_100_epochs", model=model, model_history=history)

Observations

The current neural network has three layers with 8, 4, and 1 neurons in each layer. It was trained for 100 epochs and achieved an accuracy of 0.9817, but overfitting was observed. Hence, it might be best to use 100 epochs as the optimal number for training the model. To improve the model's accuracy, we can increase the number of neurons and layers by modifying the model architecture.



Model Building with multiple layers and neurons (8-8-1) and 100 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(8, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(8, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=100, verbose=1)
Epoch 1/100
2798/2798 [==============================] - 4s 1ms/step - loss: 4.2223 - accuracy: 0.6786
Epoch 2/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.9583 - accuracy: 0.8266
Epoch 3/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.9333 - accuracy: 0.8376
Epoch 4/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.7941 - accuracy: 0.8699
Epoch 5/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6911 - accuracy: 0.8948
Epoch 6/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6511 - accuracy: 0.8972
Epoch 7/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6118 - accuracy: 0.8982
Epoch 8/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4860 - accuracy: 0.9081
Epoch 9/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3816 - accuracy: 0.9218
Epoch 10/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4414 - accuracy: 0.9173
Epoch 11/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4097 - accuracy: 0.9228
Epoch 12/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3583 - accuracy: 0.9303
Epoch 13/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3259 - accuracy: 0.9345
Epoch 14/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3331 - accuracy: 0.9346
Epoch 15/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2688 - accuracy: 0.9458
Epoch 16/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2776 - accuracy: 0.9451
Epoch 17/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2729 - accuracy: 0.9462
Epoch 18/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2514 - accuracy: 0.9503
Epoch 19/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2284 - accuracy: 0.9522
Epoch 20/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1901 - accuracy: 0.9566
Epoch 21/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1744 - accuracy: 0.9578
Epoch 22/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1753 - accuracy: 0.9569
Epoch 23/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1415 - accuracy: 0.9604
Epoch 24/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1324 - accuracy: 0.9628
Epoch 25/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1244 - accuracy: 0.9629
Epoch 26/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1199 - accuracy: 0.9632
Epoch 27/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1087 - accuracy: 0.9661
Epoch 28/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1025 - accuracy: 0.9675
Epoch 29/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1286 - accuracy: 0.9675
Epoch 30/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0955 - accuracy: 0.9708
Epoch 31/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0964 - accuracy: 0.9713
Epoch 32/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0840 - accuracy: 0.9732
Epoch 33/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0891 - accuracy: 0.9720
Epoch 34/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0866 - accuracy: 0.9727
Epoch 35/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0992 - accuracy: 0.9725
Epoch 36/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0808 - accuracy: 0.9745
Epoch 37/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0861 - accuracy: 0.9729
Epoch 38/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0809 - accuracy: 0.9734
Epoch 39/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0902 - accuracy: 0.9728
Epoch 40/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0779 - accuracy: 0.9756
Epoch 41/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0745 - accuracy: 0.9757
Epoch 42/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0874 - accuracy: 0.9729
Epoch 43/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0827 - accuracy: 0.9736
Epoch 44/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0603 - accuracy: 0.9804
Epoch 45/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0833 - accuracy: 0.9736
Epoch 46/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0612 - accuracy: 0.9818
Epoch 47/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0999 - accuracy: 0.9650
Epoch 48/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0899 - accuracy: 0.9686
Epoch 49/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1179 - accuracy: 0.9576
Epoch 50/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1411 - accuracy: 0.9537
Epoch 51/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1376 - accuracy: 0.9612
Epoch 52/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1304 - accuracy: 0.9621
Epoch 53/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1305 - accuracy: 0.9616
Epoch 54/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1304 - accuracy: 0.9619
Epoch 55/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1282 - accuracy: 0.9627
Epoch 56/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1264 - accuracy: 0.9632
Epoch 57/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1271 - accuracy: 0.9626
Epoch 58/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1267 - accuracy: 0.9628
Epoch 59/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1251 - accuracy: 0.9637
Epoch 60/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1275 - accuracy: 0.9634
Epoch 61/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1219 - accuracy: 0.9638
Epoch 62/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1208 - accuracy: 0.9652
Epoch 63/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1204 - accuracy: 0.9640
Epoch 64/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1200 - accuracy: 0.9639
Epoch 65/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1192 - accuracy: 0.9659
Epoch 66/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1147 - accuracy: 0.9670
Epoch 67/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1147 - accuracy: 0.9664
Epoch 68/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1154 - accuracy: 0.9663
Epoch 69/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1128 - accuracy: 0.9669
Epoch 70/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0957 - accuracy: 0.9728
Epoch 71/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0884 - accuracy: 0.9733
Epoch 72/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0839 - accuracy: 0.9750
Epoch 73/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0944 - accuracy: 0.9739
Epoch 74/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0851 - accuracy: 0.9745
Epoch 75/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0834 - accuracy: 0.9753
Epoch 76/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0856 - accuracy: 0.9746
Epoch 77/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0857 - accuracy: 0.9745
Epoch 78/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0937 - accuracy: 0.9752
Epoch 79/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0858 - accuracy: 0.9743
Epoch 80/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0972 - accuracy: 0.9714
Epoch 81/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0838 - accuracy: 0.9749
Epoch 82/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0807 - accuracy: 0.9753
Epoch 83/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0810 - accuracy: 0.9756
Epoch 84/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0777 - accuracy: 0.9756
Epoch 85/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0798 - accuracy: 0.9753
Epoch 86/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0796 - accuracy: 0.9743
Epoch 87/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0761 - accuracy: 0.9751
Epoch 88/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0781 - accuracy: 0.9749
Epoch 89/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0793 - accuracy: 0.9744
Epoch 90/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0781 - accuracy: 0.9749
Epoch 91/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0765 - accuracy: 0.9751
Epoch 92/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0783 - accuracy: 0.9742
Epoch 93/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0776 - accuracy: 0.9747
Epoch 94/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0750 - accuracy: 0.9754
Epoch 95/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0743 - accuracy: 0.9756
Epoch 96/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0735 - accuracy: 0.9759
Epoch 97/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0738 - accuracy: 0.9758
Epoch 98/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0803 - accuracy: 0.9758
Epoch 99/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0730 - accuracy: 0.9764
Epoch 100/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0738 - accuracy: 0.9759

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_8_8_1_100_epochs", model=model, model_history=history)

Observations

The current neural network has three layers with 8, 8, and 1 neurons in each layer. It was trained for 100 epochs and achieved an accuracy of 0.9759, but overfitting was observed. Hence, it might be best to use 100 epochs as the optimal number for training the model. To improve the model's accuracy, we can increase the number of neurons and layers by modifying the model architecture.



Model Building with multiple layers and neurons (16-8-1) and 100 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(16, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(8, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=100, verbose=1)
Epoch 1/100
2798/2798 [==============================] - 4s 1ms/step - loss: 8.3714 - accuracy: 0.8786
Epoch 2/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4498 - accuracy: 0.8856
Epoch 3/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3891 - accuracy: 0.8949
Epoch 4/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4083 - accuracy: 0.8950
Epoch 5/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3585 - accuracy: 0.8978
Epoch 6/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3355 - accuracy: 0.9003
Epoch 7/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3096 - accuracy: 0.9037
Epoch 8/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2795 - accuracy: 0.9094
Epoch 9/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2607 - accuracy: 0.9100
Epoch 10/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2508 - accuracy: 0.9103
Epoch 11/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2267 - accuracy: 0.9152
Epoch 12/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2544 - accuracy: 0.9183
Epoch 13/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2325 - accuracy: 0.9200
Epoch 14/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2298 - accuracy: 0.9287
Epoch 15/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1573 - accuracy: 0.9434
Epoch 16/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1375 - accuracy: 0.9507
Epoch 17/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1240 - accuracy: 0.9566
Epoch 18/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1134 - accuracy: 0.9605
Epoch 19/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1044 - accuracy: 0.9635
Epoch 20/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0985 - accuracy: 0.9663
Epoch 21/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1248 - accuracy: 0.9624
Epoch 22/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0968 - accuracy: 0.9671
Epoch 23/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0944 - accuracy: 0.9690
Epoch 24/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0895 - accuracy: 0.9713
Epoch 25/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0884 - accuracy: 0.9716
Epoch 26/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0832 - accuracy: 0.9730
Epoch 27/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0843 - accuracy: 0.9732
Epoch 28/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0815 - accuracy: 0.9726
Epoch 29/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0799 - accuracy: 0.9737
Epoch 30/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0779 - accuracy: 0.9743
Epoch 31/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0744 - accuracy: 0.9762
Epoch 32/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0737 - accuracy: 0.9762
Epoch 33/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0739 - accuracy: 0.9759
Epoch 34/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0746 - accuracy: 0.9756
Epoch 35/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0712 - accuracy: 0.9772
Epoch 36/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0699 - accuracy: 0.9775
Epoch 37/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0691 - accuracy: 0.9781
Epoch 38/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0699 - accuracy: 0.9779
Epoch 39/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0676 - accuracy: 0.9785
Epoch 40/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0671 - accuracy: 0.9789
Epoch 41/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0663 - accuracy: 0.9795
Epoch 42/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0667 - accuracy: 0.9796
Epoch 43/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0644 - accuracy: 0.9798
Epoch 44/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0974 - accuracy: 0.9788
Epoch 45/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0630 - accuracy: 0.9802
Epoch 46/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0642 - accuracy: 0.9803
Epoch 47/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0668 - accuracy: 0.9807
Epoch 48/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0636 - accuracy: 0.9804
Epoch 49/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0630 - accuracy: 0.9807
Epoch 50/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0608 - accuracy: 0.9814
Epoch 51/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0615 - accuracy: 0.9811
Epoch 52/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0609 - accuracy: 0.9808
Epoch 53/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0986 - accuracy: 0.9809
Epoch 54/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0616 - accuracy: 0.9804
Epoch 55/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0570 - accuracy: 0.9818
Epoch 56/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0758 - accuracy: 0.9806
Epoch 57/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0640 - accuracy: 0.9825
Epoch 58/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0586 - accuracy: 0.9816
Epoch 59/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0564 - accuracy: 0.9822
Epoch 60/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0574 - accuracy: 0.9823
Epoch 61/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0590 - accuracy: 0.9815
Epoch 62/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0569 - accuracy: 0.9820
Epoch 63/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0791 - accuracy: 0.9809
Epoch 64/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0631 - accuracy: 0.9825
Epoch 65/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0557 - accuracy: 0.9826
Epoch 66/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0556 - accuracy: 0.9826
Epoch 67/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0559 - accuracy: 0.9820
Epoch 68/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0564 - accuracy: 0.9822
Epoch 69/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0546 - accuracy: 0.9825
Epoch 70/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0535 - accuracy: 0.9831
Epoch 71/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0540 - accuracy: 0.9830
Epoch 72/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0564 - accuracy: 0.9822
Epoch 73/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0571 - accuracy: 0.9828
Epoch 74/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0595 - accuracy: 0.9827
Epoch 75/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0505 - accuracy: 0.9841
Epoch 76/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0525 - accuracy: 0.9832
Epoch 77/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0511 - accuracy: 0.9836
Epoch 78/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0523 - accuracy: 0.9835
Epoch 79/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0510 - accuracy: 0.9836
Epoch 80/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0800 - accuracy: 0.9835
Epoch 81/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0518 - accuracy: 0.9833
Epoch 82/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0501 - accuracy: 0.9837
Epoch 83/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0723 - accuracy: 0.9838
Epoch 84/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0499 - accuracy: 0.9840
Epoch 85/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0505 - accuracy: 0.9837
Epoch 86/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0495 - accuracy: 0.9840
Epoch 87/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0503 - accuracy: 0.9836
Epoch 88/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0494 - accuracy: 0.9838
Epoch 89/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0496 - accuracy: 0.9835
Epoch 90/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0488 - accuracy: 0.9841
Epoch 91/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0481 - accuracy: 0.9843
Epoch 92/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0484 - accuracy: 0.9836
Epoch 93/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0556 - accuracy: 0.9823
Epoch 94/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0437 - accuracy: 0.9854
Epoch 95/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0484 - accuracy: 0.9844
Epoch 96/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0478 - accuracy: 0.9842
Epoch 97/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0456 - accuracy: 0.9849
Epoch 98/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0455 - accuracy: 0.9848
Epoch 99/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0469 - accuracy: 0.9843
Epoch 100/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0461 - accuracy: 0.9840

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_16_8_1_100_epochs", model=model, model_history=history)

Observations

The current neural network has three layers with 16, 8, and 1 neurons in each layer. After being trained for 100 epochs, the model achieved an accuracy of 0.9840, but it showed signs of overfitting or convergence problems. As a result, it may be beneficial to stick with 100 epochs and increse the network architecture to achieve 100 percent accuracy.



Model Building with multiple layers and neurons (16-16-1) and 100 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(16, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(16, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=100, verbose=1)
Epoch 1/100
2798/2798 [==============================] - 4s 1ms/step - loss: 8.5639 - accuracy: 0.8242
Epoch 2/100
2798/2798 [==============================] - 3s 1ms/step - loss: 1.3367 - accuracy: 0.8769
Epoch 3/100
2798/2798 [==============================] - 3s 1ms/step - loss: 1.1700 - accuracy: 0.8885
Epoch 4/100
2798/2798 [==============================] - 3s 1ms/step - loss: 1.1793 - accuracy: 0.8953
Epoch 5/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.9849 - accuracy: 0.8977
Epoch 6/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8381 - accuracy: 0.9010
Epoch 7/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.7666 - accuracy: 0.9032
Epoch 8/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6470 - accuracy: 0.9073
Epoch 9/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5237 - accuracy: 0.9130
Epoch 10/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4344 - accuracy: 0.9135
Epoch 11/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3410 - accuracy: 0.9188
Epoch 12/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2856 - accuracy: 0.9187
Epoch 13/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2305 - accuracy: 0.9259
Epoch 14/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2544 - accuracy: 0.9116
Epoch 15/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2256 - accuracy: 0.9299
Epoch 16/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1926 - accuracy: 0.9290
Epoch 17/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2390 - accuracy: 0.9293
Epoch 18/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1766 - accuracy: 0.9360
Epoch 19/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1751 - accuracy: 0.9386
Epoch 20/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1754 - accuracy: 0.9413
Epoch 21/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2293 - accuracy: 0.9465
Epoch 22/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1385 - accuracy: 0.9510
Epoch 23/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1421 - accuracy: 0.9515
Epoch 24/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2056 - accuracy: 0.9523
Epoch 25/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1285 - accuracy: 0.9543
Epoch 26/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1393 - accuracy: 0.9530
Epoch 27/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1521 - accuracy: 0.9525
Epoch 28/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1176 - accuracy: 0.9568
Epoch 29/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1251 - accuracy: 0.9566
Epoch 30/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1310 - accuracy: 0.9558
Epoch 31/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1257 - accuracy: 0.9567
Epoch 32/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2513 - accuracy: 0.9577
Epoch 33/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3269 - accuracy: 0.9573
Epoch 34/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1185 - accuracy: 0.9567
Epoch 35/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1109 - accuracy: 0.9606
Epoch 36/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1225 - accuracy: 0.9599
Epoch 37/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1172 - accuracy: 0.9601
Epoch 38/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1202 - accuracy: 0.9609
Epoch 39/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1493 - accuracy: 0.9612
Epoch 40/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1060 - accuracy: 0.9626
Epoch 41/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1191 - accuracy: 0.9614
Epoch 42/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1157 - accuracy: 0.9620
Epoch 43/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1131 - accuracy: 0.9635
Epoch 44/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1894 - accuracy: 0.9640
Epoch 45/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1010 - accuracy: 0.9653
Epoch 46/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1084 - accuracy: 0.9650
Epoch 47/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1225 - accuracy: 0.9618
Epoch 48/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1125 - accuracy: 0.9651
Epoch 49/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1095 - accuracy: 0.9650
Epoch 50/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2840 - accuracy: 0.9675
Epoch 51/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1199 - accuracy: 0.9625
Epoch 52/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1004 - accuracy: 0.9685
Epoch 53/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1448 - accuracy: 0.9684
Epoch 54/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1412 - accuracy: 0.9567
Epoch 55/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1216 - accuracy: 0.9621
Epoch 56/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0966 - accuracy: 0.9691
Epoch 57/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1040 - accuracy: 0.9687
Epoch 58/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1306 - accuracy: 0.9707
Epoch 59/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0868 - accuracy: 0.9703
Epoch 60/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0853 - accuracy: 0.9718
Epoch 61/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1352 - accuracy: 0.9722
Epoch 62/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1602 - accuracy: 0.9725
Epoch 63/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0728 - accuracy: 0.9748
Epoch 64/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0805 - accuracy: 0.9725
Epoch 65/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0805 - accuracy: 0.9743
Epoch 66/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0814 - accuracy: 0.9747
Epoch 67/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0743 - accuracy: 0.9747
Epoch 68/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0736 - accuracy: 0.9752
Epoch 69/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0943 - accuracy: 0.9755
Epoch 70/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1202 - accuracy: 0.9731
Epoch 71/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0732 - accuracy: 0.9745
Epoch 72/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0709 - accuracy: 0.9744
Epoch 73/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0708 - accuracy: 0.9757
Epoch 74/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0682 - accuracy: 0.9759
Epoch 75/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0680 - accuracy: 0.9765
Epoch 76/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0690 - accuracy: 0.9775
Epoch 77/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0634 - accuracy: 0.9776
Epoch 78/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0857 - accuracy: 0.9773
Epoch 79/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0702 - accuracy: 0.9764
Epoch 80/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1094 - accuracy: 0.9769
Epoch 81/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0623 - accuracy: 0.9772
Epoch 82/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0596 - accuracy: 0.9782
Epoch 83/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0612 - accuracy: 0.9774
Epoch 84/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0645 - accuracy: 0.9762
Epoch 85/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0629 - accuracy: 0.9766
Epoch 86/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0600 - accuracy: 0.9783
Epoch 87/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0583 - accuracy: 0.9788
Epoch 88/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0588 - accuracy: 0.9782
Epoch 89/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0869 - accuracy: 0.9723
Epoch 90/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0645 - accuracy: 0.9702
Epoch 91/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0593 - accuracy: 0.9762
Epoch 92/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0575 - accuracy: 0.9791
Epoch 93/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0569 - accuracy: 0.9786
Epoch 94/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0589 - accuracy: 0.9791
Epoch 95/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0589 - accuracy: 0.9774
Epoch 96/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0539 - accuracy: 0.9798
Epoch 97/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0565 - accuracy: 0.9794
Epoch 98/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0526 - accuracy: 0.9803
Epoch 99/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0629 - accuracy: 0.9736
Epoch 100/100
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0638 - accuracy: 0.9757

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_16_16_1_100_epochs", model=model, model_history=history)

Observations

The current neural network has three layers with 16, 16, and 1 neurons in each layer. After being trained for 100 epochs, the model achieved an accuracy of 0.9759, but it showed signs of overfitting or convergence problems. As a result, it may be beneficial to stick with 100 epochs and increse the network architecture to achieve 100 percent accuracy.



Model Building with multiple layers and neurons (32-16-1) and 150 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(32, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(16, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=150, verbose=1)
Epoch 1/150
2798/2798 [==============================] - 4s 1ms/step - loss: 11.3058 - accuracy: 0.8476
Epoch 2/150
2798/2798 [==============================] - 3s 1ms/step - loss: 4.2751 - accuracy: 0.8855
Epoch 3/150
2798/2798 [==============================] - 3s 1ms/step - loss: 3.6229 - accuracy: 0.8959
Epoch 4/150
2798/2798 [==============================] - 3s 1ms/step - loss: 3.2088 - accuracy: 0.9036
Epoch 5/150
2798/2798 [==============================] - 3s 1ms/step - loss: 2.3634 - accuracy: 0.9072
Epoch 6/150
2798/2798 [==============================] - 3s 1ms/step - loss: 2.2355 - accuracy: 0.9014
Epoch 7/150
2798/2798 [==============================] - 3s 1ms/step - loss: 1.7302 - accuracy: 0.9098
Epoch 8/150
2798/2798 [==============================] - 3s 1ms/step - loss: 1.3637 - accuracy: 0.9194
Epoch 9/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.9902 - accuracy: 0.9315
Epoch 10/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8430 - accuracy: 0.9365
Epoch 11/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.7589 - accuracy: 0.9401
Epoch 12/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6123 - accuracy: 0.9468
Epoch 13/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5440 - accuracy: 0.9510
Epoch 14/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4207 - accuracy: 0.9571
Epoch 15/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3939 - accuracy: 0.9539
Epoch 16/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3855 - accuracy: 0.9485
Epoch 17/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1117 - accuracy: 0.9618
Epoch 18/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0987 - accuracy: 0.9620
Epoch 19/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0877 - accuracy: 0.9673
Epoch 20/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0819 - accuracy: 0.9695
Epoch 21/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0805 - accuracy: 0.9699
Epoch 22/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0757 - accuracy: 0.9719
Epoch 23/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0730 - accuracy: 0.9732
Epoch 24/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0719 - accuracy: 0.9736
Epoch 25/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0922 - accuracy: 0.9722
Epoch 26/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0674 - accuracy: 0.9750
Epoch 27/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0690 - accuracy: 0.9748
Epoch 28/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0684 - accuracy: 0.9750
Epoch 29/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1005 - accuracy: 0.9759
Epoch 30/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0625 - accuracy: 0.9768
Epoch 31/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0670 - accuracy: 0.9757
Epoch 32/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0622 - accuracy: 0.9771
Epoch 33/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0628 - accuracy: 0.9766
Epoch 34/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0612 - accuracy: 0.9777
Epoch 35/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0600 - accuracy: 0.9779
Epoch 36/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0668 - accuracy: 0.9763
Epoch 37/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0604 - accuracy: 0.9779
Epoch 38/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0602 - accuracy: 0.9780
Epoch 39/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0570 - accuracy: 0.9791
Epoch 40/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1306 - accuracy: 0.9769
Epoch 41/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0646 - accuracy: 0.9778
Epoch 42/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0521 - accuracy: 0.9807
Epoch 43/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0529 - accuracy: 0.9806
Epoch 44/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0505 - accuracy: 0.9815
Epoch 45/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0492 - accuracy: 0.9819
Epoch 46/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0533 - accuracy: 0.9807
Epoch 47/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0491 - accuracy: 0.9819
Epoch 48/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0807 - accuracy: 0.9784
Epoch 49/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0740 - accuracy: 0.9793
Epoch 50/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0442 - accuracy: 0.9839
Epoch 51/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0449 - accuracy: 0.9842
Epoch 52/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0441 - accuracy: 0.9838
Epoch 53/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0485 - accuracy: 0.9837
Epoch 54/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0426 - accuracy: 0.9846
Epoch 55/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0430 - accuracy: 0.9848
Epoch 56/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0432 - accuracy: 0.9849
Epoch 57/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0424 - accuracy: 0.9851
Epoch 58/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0455 - accuracy: 0.9845
Epoch 59/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1906 - accuracy: 0.9835
Epoch 60/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0410 - accuracy: 0.9858
Epoch 61/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0396 - accuracy: 0.9863
Epoch 62/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0434 - accuracy: 0.9848
Epoch 63/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0399 - accuracy: 0.9856
Epoch 64/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0399 - accuracy: 0.9860
Epoch 65/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0405 - accuracy: 0.9855
Epoch 66/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0392 - accuracy: 0.9862
Epoch 67/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0402 - accuracy: 0.9862
Epoch 68/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0395 - accuracy: 0.9863
Epoch 69/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0381 - accuracy: 0.9865
Epoch 70/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1198 - accuracy: 0.9857
Epoch 71/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0637 - accuracy: 0.9846
Epoch 72/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0593 - accuracy: 0.9848
Epoch 73/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0400 - accuracy: 0.9868
Epoch 74/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0389 - accuracy: 0.9869
Epoch 75/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0364 - accuracy: 0.9878
Epoch 76/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0368 - accuracy: 0.9875
Epoch 77/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0358 - accuracy: 0.9877
Epoch 78/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0342 - accuracy: 0.9884
Epoch 79/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0376 - accuracy: 0.9872
Epoch 80/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0564 - accuracy: 0.9876
Epoch 81/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0351 - accuracy: 0.9885
Epoch 82/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0338 - accuracy: 0.9884
Epoch 83/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0323 - accuracy: 0.9892
Epoch 84/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0372 - accuracy: 0.9877
Epoch 85/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0324 - accuracy: 0.9891
Epoch 86/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0324 - accuracy: 0.9886
Epoch 87/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0313 - accuracy: 0.9895
Epoch 88/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0316 - accuracy: 0.9889
Epoch 89/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0317 - accuracy: 0.9892
Epoch 90/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0325 - accuracy: 0.9885
Epoch 91/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0330 - accuracy: 0.9888
Epoch 92/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0312 - accuracy: 0.9890
Epoch 93/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0300 - accuracy: 0.9895
Epoch 94/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0420 - accuracy: 0.9879
Epoch 95/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0625 - accuracy: 0.9882
Epoch 96/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0327 - accuracy: 0.9889
Epoch 97/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0336 - accuracy: 0.9884
Epoch 98/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0316 - accuracy: 0.9891
Epoch 99/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0313 - accuracy: 0.9889
Epoch 100/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0307 - accuracy: 0.9894
Epoch 101/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0300 - accuracy: 0.9893
Epoch 102/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0307 - accuracy: 0.9889
Epoch 103/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0306 - accuracy: 0.9889
Epoch 104/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0298 - accuracy: 0.9896
Epoch 105/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0291 - accuracy: 0.9900
Epoch 106/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0278 - accuracy: 0.9901
Epoch 107/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0306 - accuracy: 0.9891
Epoch 108/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0274 - accuracy: 0.9905
Epoch 109/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0447 - accuracy: 0.9873
Epoch 110/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0281 - accuracy: 0.9899
Epoch 111/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0277 - accuracy: 0.9902
Epoch 112/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0302 - accuracy: 0.9894
Epoch 113/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0311 - accuracy: 0.9891
Epoch 114/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0314 - accuracy: 0.9891
Epoch 115/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0318 - accuracy: 0.9894
Epoch 116/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0284 - accuracy: 0.9900
Epoch 117/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0292 - accuracy: 0.9898
Epoch 118/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0288 - accuracy: 0.9895
Epoch 119/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0293 - accuracy: 0.9897
Epoch 120/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0282 - accuracy: 0.9899
Epoch 121/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0287 - accuracy: 0.9902
Epoch 122/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0277 - accuracy: 0.9901
Epoch 123/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0521 - accuracy: 0.9857
Epoch 124/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0601 - accuracy: 0.9885
Epoch 125/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0690 - accuracy: 0.9890
Epoch 126/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0337 - accuracy: 0.9892
Epoch 127/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0316 - accuracy: 0.9884
Epoch 128/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0289 - accuracy: 0.9899
Epoch 129/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0305 - accuracy: 0.9894
Epoch 130/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0410 - accuracy: 0.9906
Epoch 131/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0274 - accuracy: 0.9902
Epoch 132/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0276 - accuracy: 0.9901
Epoch 133/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0276 - accuracy: 0.9902
Epoch 134/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0271 - accuracy: 0.9902
Epoch 135/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0278 - accuracy: 0.9900
Epoch 136/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0276 - accuracy: 0.9904
Epoch 137/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0285 - accuracy: 0.9902
Epoch 138/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0273 - accuracy: 0.9901
Epoch 139/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0268 - accuracy: 0.9906
Epoch 140/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0284 - accuracy: 0.9897
Epoch 141/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0641 - accuracy: 0.9901
Epoch 142/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0265 - accuracy: 0.9907
Epoch 143/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0268 - accuracy: 0.9903
Epoch 144/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0301 - accuracy: 0.9896
Epoch 145/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0270 - accuracy: 0.9903
Epoch 146/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0281 - accuracy: 0.9898
Epoch 147/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0273 - accuracy: 0.9904
Epoch 148/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0269 - accuracy: 0.9901
Epoch 149/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0270 - accuracy: 0.9908
Epoch 150/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0255 - accuracy: 0.9905

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_32_16_1_150_epochs", model=model, model_history=history)

Observations

The current neural network has three layers with 16, 16, and 1 neurons in each layer. After being trained for 100 epochs, the model achieved an accuracy of 0.9905, but it showed signs of overfitting or convergence problems. As a result, it may be beneficial to stick with 150 epochs and increse the network architecture to achieve 100 percent accuracy.



Model Building with multiple layers and neurons (32-32-1) and 150 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(32, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(32, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=150, verbose=1)
Epoch 1/150
2798/2798 [==============================] - 4s 1ms/step - loss: 6.6292 - accuracy: 0.8412
Epoch 2/150
2798/2798 [==============================] - 3s 1ms/step - loss: 2.7904 - accuracy: 0.8877
Epoch 3/150
2798/2798 [==============================] - 3s 1ms/step - loss: 2.0039 - accuracy: 0.9092
Epoch 4/150
2798/2798 [==============================] - 3s 1ms/step - loss: 1.3562 - accuracy: 0.9285
Epoch 5/150
2798/2798 [==============================] - 3s 1ms/step - loss: 1.2621 - accuracy: 0.9308
Epoch 6/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6961 - accuracy: 0.9469
Epoch 7/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5869 - accuracy: 0.9508
Epoch 8/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4978 - accuracy: 0.9570
Epoch 9/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3711 - accuracy: 0.9570
Epoch 10/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2806 - accuracy: 0.9612
Epoch 11/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2504 - accuracy: 0.9605
Epoch 12/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1414 - accuracy: 0.9646
Epoch 13/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1087 - accuracy: 0.9680
Epoch 14/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0938 - accuracy: 0.9724
Epoch 15/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0803 - accuracy: 0.9764
Epoch 16/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0732 - accuracy: 0.9764
Epoch 17/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0573 - accuracy: 0.9807
Epoch 18/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0618 - accuracy: 0.9794
Epoch 19/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0689 - accuracy: 0.9775
Epoch 20/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0675 - accuracy: 0.9739
Epoch 21/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0624 - accuracy: 0.9759
Epoch 22/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0687 - accuracy: 0.9741
Epoch 23/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0687 - accuracy: 0.9745
Epoch 24/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0584 - accuracy: 0.9770
Epoch 25/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0575 - accuracy: 0.9780
Epoch 26/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0812 - accuracy: 0.9768
Epoch 27/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0814 - accuracy: 0.9783
Epoch 28/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0803 - accuracy: 0.9797
Epoch 29/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0523 - accuracy: 0.9826
Epoch 30/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0513 - accuracy: 0.9823
Epoch 31/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0490 - accuracy: 0.9834
Epoch 32/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1354 - accuracy: 0.9831
Epoch 33/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0503 - accuracy: 0.9834
Epoch 34/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0521 - accuracy: 0.9829
Epoch 35/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0498 - accuracy: 0.9838
Epoch 36/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0442 - accuracy: 0.9851
Epoch 37/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0542 - accuracy: 0.9815
Epoch 38/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0862 - accuracy: 0.9717
Epoch 39/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2338 - accuracy: 0.9161
Epoch 40/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2131 - accuracy: 0.9217
Epoch 41/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1338 - accuracy: 0.9519
Epoch 42/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0635 - accuracy: 0.9796
Epoch 43/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0463 - accuracy: 0.9844
Epoch 44/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0539 - accuracy: 0.9835
Epoch 45/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0753 - accuracy: 0.9758
Epoch 46/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0629 - accuracy: 0.9802
Epoch 47/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0640 - accuracy: 0.9807
Epoch 48/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0561 - accuracy: 0.9817
Epoch 49/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0956 - accuracy: 0.9655
Epoch 50/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1539 - accuracy: 0.9332
Epoch 51/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1634 - accuracy: 0.9349
Epoch 52/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1365 - accuracy: 0.9368
Epoch 53/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1333 - accuracy: 0.9387
Epoch 54/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1282 - accuracy: 0.9409
Epoch 55/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1239 - accuracy: 0.9437
Epoch 56/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1256 - accuracy: 0.9453
Epoch 57/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1339 - accuracy: 0.9479
Epoch 58/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1179 - accuracy: 0.9498
Epoch 59/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1188 - accuracy: 0.9509
Epoch 60/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1126 - accuracy: 0.9540
Epoch 61/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1161 - accuracy: 0.9529
Epoch 62/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1120 - accuracy: 0.9543
Epoch 63/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1319 - accuracy: 0.9453
Epoch 64/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1086 - accuracy: 0.9552
Epoch 65/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1086 - accuracy: 0.9549
Epoch 66/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1092 - accuracy: 0.9556
Epoch 67/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1330 - accuracy: 0.9522
Epoch 68/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1085 - accuracy: 0.9570
Epoch 69/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1235 - accuracy: 0.9477
Epoch 70/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1076 - accuracy: 0.9552
Epoch 71/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1611 - accuracy: 0.9409
Epoch 72/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1349 - accuracy: 0.9470
Epoch 73/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1386 - accuracy: 0.9520
Epoch 74/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1025 - accuracy: 0.9578
Epoch 75/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1835 - accuracy: 0.9332
Epoch 76/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1906 - accuracy: 0.9332
Epoch 77/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1450 - accuracy: 0.9410
Epoch 78/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1045 - accuracy: 0.9579
Epoch 79/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1035 - accuracy: 0.9583
Epoch 80/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1086 - accuracy: 0.9573
Epoch 81/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1053 - accuracy: 0.9590
Epoch 82/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1049 - accuracy: 0.9583
Epoch 83/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1679 - accuracy: 0.9433
Epoch 84/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1782 - accuracy: 0.9430
Epoch 85/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1743 - accuracy: 0.9412
Epoch 86/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1868 - accuracy: 0.9305
Epoch 87/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1769 - accuracy: 0.9231
Epoch 88/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1119 - accuracy: 0.9567
Epoch 89/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1110 - accuracy: 0.9579
Epoch 90/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1070 - accuracy: 0.9597
Epoch 91/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0979 - accuracy: 0.9605
Epoch 92/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1247 - accuracy: 0.9562
Epoch 93/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0997 - accuracy: 0.9603
Epoch 94/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1619 - accuracy: 0.9502
Epoch 95/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1489 - accuracy: 0.9509
Epoch 96/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1512 - accuracy: 0.9484
Epoch 97/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1459 - accuracy: 0.9498
Epoch 98/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1235 - accuracy: 0.9548
Epoch 99/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1270 - accuracy: 0.9583
Epoch 100/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1118 - accuracy: 0.9585
Epoch 101/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1270 - accuracy: 0.9569
Epoch 102/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1158 - accuracy: 0.9587
Epoch 103/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1242 - accuracy: 0.9577
Epoch 104/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1304 - accuracy: 0.9607
Epoch 105/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1024 - accuracy: 0.9609
Epoch 106/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1071 - accuracy: 0.9600
Epoch 107/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1027 - accuracy: 0.9612
Epoch 108/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1052 - accuracy: 0.9601
Epoch 109/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1105 - accuracy: 0.9601
Epoch 110/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0957 - accuracy: 0.9635
Epoch 111/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1020 - accuracy: 0.9615
Epoch 112/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1011 - accuracy: 0.9624
Epoch 113/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0991 - accuracy: 0.9632
Epoch 114/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0983 - accuracy: 0.9622
Epoch 115/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0965 - accuracy: 0.9626
Epoch 116/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1237 - accuracy: 0.9620
Epoch 117/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1010 - accuracy: 0.9623
Epoch 118/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0995 - accuracy: 0.9618
Epoch 119/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1071 - accuracy: 0.9609
Epoch 120/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1006 - accuracy: 0.9618
Epoch 121/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1442 - accuracy: 0.9527
Epoch 122/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1585 - accuracy: 0.9530
Epoch 123/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2386 - accuracy: 0.8862
Epoch 124/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6402 - accuracy: 0.5741
Epoch 125/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3550 - accuracy: 0.8188
Epoch 126/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1726 - accuracy: 0.9356
Epoch 127/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1448 - accuracy: 0.9518
Epoch 128/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1483 - accuracy: 0.9511
Epoch 129/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1426 - accuracy: 0.9533
Epoch 130/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1298 - accuracy: 0.9563
Epoch 131/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1255 - accuracy: 0.9571
Epoch 132/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1214 - accuracy: 0.9584
Epoch 133/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1301 - accuracy: 0.9538
Epoch 134/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1111 - accuracy: 0.9615
Epoch 135/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1446 - accuracy: 0.9504
Epoch 136/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1180 - accuracy: 0.9587
Epoch 137/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1029 - accuracy: 0.9642
Epoch 138/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1000 - accuracy: 0.9655
Epoch 139/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0921 - accuracy: 0.9687
Epoch 140/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1350 - accuracy: 0.9537
Epoch 141/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0986 - accuracy: 0.9668
Epoch 142/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0862 - accuracy: 0.9712
Epoch 143/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0782 - accuracy: 0.9739
Epoch 144/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0825 - accuracy: 0.9726
Epoch 145/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0953 - accuracy: 0.9742
Epoch 146/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0906 - accuracy: 0.9723
Epoch 147/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0770 - accuracy: 0.9748
Epoch 148/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0823 - accuracy: 0.9730
Epoch 149/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0782 - accuracy: 0.9741
Epoch 150/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0830 - accuracy: 0.9734

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_32_32_1_150_epochs", model=model, model_history=history)

Observations

The current neural network has three layers with 32, 32, and 1 neurons in each layer. After being trained for 100 epochs, the model achieved an accuracy of 0.9734, but it showed signs of overfitting or convergence problems. As a result, it may be beneficial to stick with 150 epochs and increse the network architecture to achieve 100 percent accuracy.



Model Building with multiple layers and neurons (128-128-1) and 150 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(128, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(128, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=150, verbose=1)
Epoch 1/150
2798/2798 [==============================] - 5s 1ms/step - loss: 13.4671 - accuracy: 0.8437
Epoch 2/150
2798/2798 [==============================] - 3s 1ms/step - loss: 5.3187 - accuracy: 0.8845
Epoch 3/150
2798/2798 [==============================] - 3s 1ms/step - loss: 2.5273 - accuracy: 0.9013
Epoch 4/150
2798/2798 [==============================] - 3s 1ms/step - loss: 1.0841 - accuracy: 0.9174
Epoch 5/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8285 - accuracy: 0.9282
Epoch 6/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2499 - accuracy: 0.9448
Epoch 7/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2451 - accuracy: 0.9567
Epoch 8/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0963 - accuracy: 0.9654
Epoch 9/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0855 - accuracy: 0.9697
Epoch 10/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0723 - accuracy: 0.9743
Epoch 11/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0707 - accuracy: 0.9757
Epoch 12/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0604 - accuracy: 0.9781
Epoch 13/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0510 - accuracy: 0.9804
Epoch 14/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0477 - accuracy: 0.9820
Epoch 15/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0495 - accuracy: 0.9814
Epoch 16/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0484 - accuracy: 0.9818
Epoch 17/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0459 - accuracy: 0.9829
Epoch 18/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0473 - accuracy: 0.9825
Epoch 19/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0450 - accuracy: 0.9836
Epoch 20/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0463 - accuracy: 0.9828
Epoch 21/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0445 - accuracy: 0.9837
Epoch 22/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0410 - accuracy: 0.9846
Epoch 23/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0409 - accuracy: 0.9843
Epoch 24/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0572 - accuracy: 0.9839
Epoch 25/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0433 - accuracy: 0.9833
Epoch 26/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0386 - accuracy: 0.9845
Epoch 27/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0453 - accuracy: 0.9832
Epoch 28/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0441 - accuracy: 0.9841
Epoch 29/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0428 - accuracy: 0.9844
Epoch 30/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0445 - accuracy: 0.9843
Epoch 31/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0448 - accuracy: 0.9839
Epoch 32/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0418 - accuracy: 0.9846
Epoch 33/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0426 - accuracy: 0.9844
Epoch 34/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0412 - accuracy: 0.9849
Epoch 35/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0411 - accuracy: 0.9849
Epoch 36/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0403 - accuracy: 0.9852
Epoch 37/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0406 - accuracy: 0.9846
Epoch 38/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0387 - accuracy: 0.9855
Epoch 39/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0405 - accuracy: 0.9854
Epoch 40/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0385 - accuracy: 0.9861
Epoch 41/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0367 - accuracy: 0.9861
Epoch 42/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0387 - accuracy: 0.9859
Epoch 43/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0790 - accuracy: 0.9856
Epoch 44/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0381 - accuracy: 0.9860
Epoch 45/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0384 - accuracy: 0.9859
Epoch 46/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0360 - accuracy: 0.9861
Epoch 47/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0389 - accuracy: 0.9859
Epoch 48/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0385 - accuracy: 0.9863
Epoch 49/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0360 - accuracy: 0.9862
Epoch 50/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0377 - accuracy: 0.9857
Epoch 51/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0374 - accuracy: 0.9870
Epoch 52/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0381 - accuracy: 0.9862
Epoch 53/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0359 - accuracy: 0.9873
Epoch 54/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0363 - accuracy: 0.9867
Epoch 55/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1298 - accuracy: 0.9856
Epoch 56/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0363 - accuracy: 0.9857
Epoch 57/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0355 - accuracy: 0.9862
Epoch 58/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0382 - accuracy: 0.9870
Epoch 59/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0339 - accuracy: 0.9877
Epoch 60/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0378 - accuracy: 0.9859
Epoch 61/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0332 - accuracy: 0.9876
Epoch 62/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0342 - accuracy: 0.9870
Epoch 63/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0374 - accuracy: 0.9864
Epoch 64/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0349 - accuracy: 0.9874
Epoch 65/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0332 - accuracy: 0.9877
Epoch 66/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0340 - accuracy: 0.9870
Epoch 67/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0354 - accuracy: 0.9875
Epoch 68/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0332 - accuracy: 0.9878
Epoch 69/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0342 - accuracy: 0.9873
Epoch 70/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0356 - accuracy: 0.9869
Epoch 71/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0342 - accuracy: 0.9873
Epoch 72/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0332 - accuracy: 0.9877
Epoch 73/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0338 - accuracy: 0.9875
Epoch 74/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0335 - accuracy: 0.9876
Epoch 75/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0343 - accuracy: 0.9872
Epoch 76/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0320 - accuracy: 0.9879
Epoch 77/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0325 - accuracy: 0.9881
Epoch 78/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0341 - accuracy: 0.9872
Epoch 79/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0316 - accuracy: 0.9882
Epoch 80/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0332 - accuracy: 0.9880
Epoch 81/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0322 - accuracy: 0.9882
Epoch 82/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0351 - accuracy: 0.9874
Epoch 83/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0356 - accuracy: 0.9869
Epoch 84/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0314 - accuracy: 0.9883
Epoch 85/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0342 - accuracy: 0.9871
Epoch 86/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0332 - accuracy: 0.9876
Epoch 87/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0316 - accuracy: 0.9886
Epoch 88/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0316 - accuracy: 0.9882
Epoch 89/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0308 - accuracy: 0.9886
Epoch 90/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0323 - accuracy: 0.9879
Epoch 91/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0325 - accuracy: 0.9879
Epoch 92/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0348 - accuracy: 0.9870
Epoch 93/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0310 - accuracy: 0.9880
Epoch 94/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0322 - accuracy: 0.9878
Epoch 95/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0314 - accuracy: 0.9880
Epoch 96/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1465 - accuracy: 0.9876
Epoch 97/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0298 - accuracy: 0.9885
Epoch 98/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0316 - accuracy: 0.9879
Epoch 99/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0315 - accuracy: 0.9886
Epoch 100/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0314 - accuracy: 0.9876
Epoch 101/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0318 - accuracy: 0.9882
Epoch 102/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0314 - accuracy: 0.9887
Epoch 103/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0308 - accuracy: 0.9890
Epoch 104/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0313 - accuracy: 0.9885
Epoch 105/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0318 - accuracy: 0.9881
Epoch 106/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0311 - accuracy: 0.9883
Epoch 107/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0302 - accuracy: 0.9886
Epoch 108/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0308 - accuracy: 0.9882
Epoch 109/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0304 - accuracy: 0.9890
Epoch 110/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0312 - accuracy: 0.9889
Epoch 111/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0314 - accuracy: 0.9880
Epoch 112/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0292 - accuracy: 0.9889
Epoch 113/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0301 - accuracy: 0.9886
Epoch 114/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0290 - accuracy: 0.9891
Epoch 115/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0333 - accuracy: 0.9881
Epoch 116/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0301 - accuracy: 0.9886
Epoch 117/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0345 - accuracy: 0.9878
Epoch 118/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0530 - accuracy: 0.9821
Epoch 119/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1500 - accuracy: 0.9836
Epoch 120/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0767 - accuracy: 0.9858
Epoch 121/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1066 - accuracy: 0.9851
Epoch 122/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0382 - accuracy: 0.9861
Epoch 123/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0360 - accuracy: 0.9868
Epoch 124/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0368 - accuracy: 0.9870
Epoch 125/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0378 - accuracy: 0.9866
Epoch 126/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0336 - accuracy: 0.9878
Epoch 127/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0355 - accuracy: 0.9875
Epoch 128/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0336 - accuracy: 0.9876
Epoch 129/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0346 - accuracy: 0.9874
Epoch 130/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0347 - accuracy: 0.9876
Epoch 131/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0348 - accuracy: 0.9875
Epoch 132/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0336 - accuracy: 0.9874
Epoch 133/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0362 - accuracy: 0.9865
Epoch 134/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0319 - accuracy: 0.9882
Epoch 135/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0324 - accuracy: 0.9880
Epoch 136/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0317 - accuracy: 0.9884
Epoch 137/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0335 - accuracy: 0.9878
Epoch 138/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0353 - accuracy: 0.9875
Epoch 139/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0362 - accuracy: 0.9878
Epoch 140/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0328 - accuracy: 0.9880
Epoch 141/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0346 - accuracy: 0.9873
Epoch 142/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0334 - accuracy: 0.9878
Epoch 143/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0344 - accuracy: 0.9876
Epoch 144/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0328 - accuracy: 0.9881
Epoch 145/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0330 - accuracy: 0.9882
Epoch 146/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0330 - accuracy: 0.9878
Epoch 147/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0339 - accuracy: 0.9875
Epoch 148/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0336 - accuracy: 0.9879
Epoch 149/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0331 - accuracy: 0.9876
Epoch 150/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0307 - accuracy: 0.9885

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_128_128_1_150_epochs", model=model, model_history=history)

Observations

The current neural network has four layers with 128, 128 and 1 neurons in each layer. After being trained for 100 epochs, the model achieved an accuracy of 0.9885, it showed signs of overfitting or convergence. As a result, it may be beneficial to stick with 100 epochs.



Model Building with multiple layers and neurons (8-8-8-1) and 150 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(8, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(8, activation="relu"))
model.add(Dense(8, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=150, verbose=1)
Epoch 1/150
2798/2798 [==============================] - 4s 1ms/step - loss: 56.6353 - accuracy: 0.6884
Epoch 2/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4000 - accuracy: 0.8627
Epoch 3/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3008 - accuracy: 0.8946
Epoch 4/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2896 - accuracy: 0.9052
Epoch 5/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2513 - accuracy: 0.9139
Epoch 6/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2302 - accuracy: 0.9143
Epoch 7/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2267 - accuracy: 0.9162
Epoch 8/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2209 - accuracy: 0.9177
Epoch 9/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2170 - accuracy: 0.9189
Epoch 10/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2072 - accuracy: 0.9203
Epoch 11/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1967 - accuracy: 0.9232
Epoch 12/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2358 - accuracy: 0.9248
Epoch 13/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1912 - accuracy: 0.9246
Epoch 14/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1836 - accuracy: 0.9267
Epoch 15/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1837 - accuracy: 0.9270
Epoch 16/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1791 - accuracy: 0.9286
Epoch 17/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1685 - accuracy: 0.9364
Epoch 18/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2107 - accuracy: 0.9426
Epoch 19/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1461 - accuracy: 0.9466
Epoch 20/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1307 - accuracy: 0.9507
Epoch 21/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1204 - accuracy: 0.9555
Epoch 22/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1132 - accuracy: 0.9596
Epoch 23/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1263 - accuracy: 0.9651
Epoch 24/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1268 - accuracy: 0.9665
Epoch 25/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0965 - accuracy: 0.9681
Epoch 26/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0919 - accuracy: 0.9703
Epoch 27/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0882 - accuracy: 0.9714
Epoch 28/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0874 - accuracy: 0.9718
Epoch 29/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1073 - accuracy: 0.9726
Epoch 30/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0844 - accuracy: 0.9730
Epoch 31/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0822 - accuracy: 0.9743
Epoch 32/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0845 - accuracy: 0.9737
Epoch 33/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0827 - accuracy: 0.9748
Epoch 34/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0850 - accuracy: 0.9740
Epoch 35/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0797 - accuracy: 0.9752
Epoch 36/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3686 - accuracy: 0.9082
Epoch 37/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3787 - accuracy: 0.8598
Epoch 38/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3348 - accuracy: 0.8596
Epoch 39/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1814 - accuracy: 0.9424
Epoch 40/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0870 - accuracy: 0.9744
Epoch 41/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0760 - accuracy: 0.9756
Epoch 42/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0969 - accuracy: 0.9753
Epoch 43/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0665 - accuracy: 0.9773
Epoch 44/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0660 - accuracy: 0.9777
Epoch 45/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0663 - accuracy: 0.9770
Epoch 46/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0815 - accuracy: 0.9773
Epoch 47/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0636 - accuracy: 0.9775
Epoch 48/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0640 - accuracy: 0.9778
Epoch 49/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0644 - accuracy: 0.9776
Epoch 50/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0623 - accuracy: 0.9776
Epoch 51/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0619 - accuracy: 0.9782
Epoch 52/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0620 - accuracy: 0.9784
Epoch 53/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0602 - accuracy: 0.9786
Epoch 54/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0608 - accuracy: 0.9790
Epoch 55/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0800 - accuracy: 0.9791
Epoch 56/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0589 - accuracy: 0.9788
Epoch 57/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0590 - accuracy: 0.9791
Epoch 58/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0611 - accuracy: 0.9781
Epoch 59/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0592 - accuracy: 0.9793
Epoch 60/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0580 - accuracy: 0.9797
Epoch 61/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0733 - accuracy: 0.9807
Epoch 62/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0547 - accuracy: 0.9801
Epoch 63/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0548 - accuracy: 0.9804
Epoch 64/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0604 - accuracy: 0.9783
Epoch 65/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0564 - accuracy: 0.9802
Epoch 66/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0546 - accuracy: 0.9800
Epoch 67/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0545 - accuracy: 0.9801
Epoch 68/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0578 - accuracy: 0.9787
Epoch 69/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0604 - accuracy: 0.9809
Epoch 70/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0543 - accuracy: 0.9803
Epoch 71/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0529 - accuracy: 0.9812
Epoch 72/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0637 - accuracy: 0.9809
Epoch 73/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0520 - accuracy: 0.9813
Epoch 74/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0521 - accuracy: 0.9811
Epoch 75/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0696 - accuracy: 0.9810
Epoch 76/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0599 - accuracy: 0.9796
Epoch 77/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0508 - accuracy: 0.9817
Epoch 78/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0515 - accuracy: 0.9815
Epoch 79/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0508 - accuracy: 0.9815
Epoch 80/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0486 - accuracy: 0.9826
Epoch 81/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0569 - accuracy: 0.9827
Epoch 82/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0410 - accuracy: 0.9863
Epoch 83/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0404 - accuracy: 0.9864
Epoch 84/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0441 - accuracy: 0.9853
Epoch 85/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0413 - accuracy: 0.9860
Epoch 86/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0483 - accuracy: 0.9850
Epoch 87/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0401 - accuracy: 0.9865
Epoch 88/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0403 - accuracy: 0.9865
Epoch 89/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0718 - accuracy: 0.9723
Epoch 90/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0547 - accuracy: 0.9795
Epoch 91/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0522 - accuracy: 0.9804
Epoch 92/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0517 - accuracy: 0.9810
Epoch 93/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0505 - accuracy: 0.9817
Epoch 94/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0494 - accuracy: 0.9819
Epoch 95/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0487 - accuracy: 0.9819
Epoch 96/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0537 - accuracy: 0.9809
Epoch 97/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0487 - accuracy: 0.9822
Epoch 98/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0482 - accuracy: 0.9826
Epoch 99/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0486 - accuracy: 0.9821
Epoch 100/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0461 - accuracy: 0.9827
Epoch 101/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0495 - accuracy: 0.9819
Epoch 102/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0465 - accuracy: 0.9824
Epoch 103/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0581 - accuracy: 0.9825
Epoch 104/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0465 - accuracy: 0.9830
Epoch 105/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0466 - accuracy: 0.9827
Epoch 106/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0528 - accuracy: 0.9828
Epoch 107/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0446 - accuracy: 0.9835
Epoch 108/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0458 - accuracy: 0.9831
Epoch 109/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0459 - accuracy: 0.9832
Epoch 110/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0444 - accuracy: 0.9840
Epoch 111/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0468 - accuracy: 0.9828
Epoch 112/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0633 - accuracy: 0.9812
Epoch 113/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0427 - accuracy: 0.9843
Epoch 114/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0445 - accuracy: 0.9839
Epoch 115/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0449 - accuracy: 0.9836
Epoch 116/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0715 - accuracy: 0.9804
Epoch 117/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0441 - accuracy: 0.9840
Epoch 118/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0457 - accuracy: 0.9831
Epoch 119/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0444 - accuracy: 0.9837
Epoch 120/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0437 - accuracy: 0.9837
Epoch 121/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0421 - accuracy: 0.9840
Epoch 122/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0425 - accuracy: 0.9840
Epoch 123/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0480 - accuracy: 0.9826
Epoch 124/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0426 - accuracy: 0.9838
Epoch 125/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0465 - accuracy: 0.9823
Epoch 126/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0426 - accuracy: 0.9839
Epoch 127/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0430 - accuracy: 0.9839
Epoch 128/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0430 - accuracy: 0.9837
Epoch 129/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0435 - accuracy: 0.9840
Epoch 130/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0419 - accuracy: 0.9844
Epoch 131/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0423 - accuracy: 0.9843
Epoch 132/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0423 - accuracy: 0.9839
Epoch 133/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0417 - accuracy: 0.9842
Epoch 134/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0409 - accuracy: 0.9844
Epoch 135/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0400 - accuracy: 0.9846
Epoch 136/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0415 - accuracy: 0.9842
Epoch 137/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0412 - accuracy: 0.9843
Epoch 138/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0408 - accuracy: 0.9844
Epoch 139/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0433 - accuracy: 0.9840
Epoch 140/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0395 - accuracy: 0.9853
Epoch 141/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0402 - accuracy: 0.9844
Epoch 142/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0501 - accuracy: 0.9801
Epoch 143/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0416 - accuracy: 0.9839
Epoch 144/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0421 - accuracy: 0.9842
Epoch 145/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0412 - accuracy: 0.9846
Epoch 146/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0444 - accuracy: 0.9836
Epoch 147/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0404 - accuracy: 0.9842
Epoch 148/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0412 - accuracy: 0.9847
Epoch 149/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0413 - accuracy: 0.9847
Epoch 150/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0444 - accuracy: 0.9835

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_8_8_8_1_150_epochs", model=model, model_history=history)

Observations

The current neural network has four layers with 32, 16, 8 and 1 neurons in each layer. After being trained for 100 epochs, the model achieved an accuracy of 0.9835, it showed signs of overfitting or convergence. As a result, it may be beneficial to stick with 100 epochs.



Model Building with multiple layers and neurons (16-8-16-1) and 150 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(16, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(8, activation="relu"))
model.add(Dense(16, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=150, verbose=1)
Epoch 1/150
2798/2798 [==============================] - 4s 1ms/step - loss: 2.5886 - accuracy: 0.8218
Epoch 2/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.7275 - accuracy: 0.8649
Epoch 3/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6186 - accuracy: 0.8883
Epoch 4/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6615 - accuracy: 0.8907
Epoch 5/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5693 - accuracy: 0.9004
Epoch 6/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4500 - accuracy: 0.9132
Epoch 7/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3474 - accuracy: 0.9257
Epoch 8/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2489 - accuracy: 0.9377
Epoch 9/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1765 - accuracy: 0.9501
Epoch 10/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1332 - accuracy: 0.9575
Epoch 11/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1168 - accuracy: 0.9570
Epoch 12/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1023 - accuracy: 0.9625
Epoch 13/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1492 - accuracy: 0.9518
Epoch 14/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1569 - accuracy: 0.9456
Epoch 15/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1317 - accuracy: 0.9524
Epoch 16/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1244 - accuracy: 0.9545
Epoch 17/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1207 - accuracy: 0.9562
Epoch 18/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1070 - accuracy: 0.9620
Epoch 19/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1494 - accuracy: 0.9517
Epoch 20/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1156 - accuracy: 0.9622
Epoch 21/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1114 - accuracy: 0.9618
Epoch 22/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1123 - accuracy: 0.9595
Epoch 23/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1063 - accuracy: 0.9624
Epoch 24/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1162 - accuracy: 0.9613
Epoch 25/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1252 - accuracy: 0.9580
Epoch 26/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1142 - accuracy: 0.9625
Epoch 27/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1100 - accuracy: 0.9630
Epoch 28/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1091 - accuracy: 0.9622
Epoch 29/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1297 - accuracy: 0.9561
Epoch 30/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1010 - accuracy: 0.9659
Epoch 31/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1065 - accuracy: 0.9651
Epoch 32/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1056 - accuracy: 0.9645
Epoch 33/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0998 - accuracy: 0.9662
Epoch 34/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0898 - accuracy: 0.9701
Epoch 35/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1007 - accuracy: 0.9673
Epoch 36/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0998 - accuracy: 0.9665
Epoch 37/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1053 - accuracy: 0.9658
Epoch 38/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0920 - accuracy: 0.9694
Epoch 39/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0951 - accuracy: 0.9695
Epoch 40/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1043 - accuracy: 0.9652
Epoch 41/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0911 - accuracy: 0.9707
Epoch 42/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0860 - accuracy: 0.9704
Epoch 43/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0931 - accuracy: 0.9684
Epoch 44/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1036 - accuracy: 0.9680
Epoch 45/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0960 - accuracy: 0.9671
Epoch 46/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0940 - accuracy: 0.9680
Epoch 47/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0900 - accuracy: 0.9699
Epoch 48/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0904 - accuracy: 0.9698
Epoch 49/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0747 - accuracy: 0.9739
Epoch 50/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0850 - accuracy: 0.9712
Epoch 51/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0736 - accuracy: 0.9746
Epoch 52/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0864 - accuracy: 0.9718
Epoch 53/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0711 - accuracy: 0.9762
Epoch 54/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0782 - accuracy: 0.9728
Epoch 55/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1324 - accuracy: 0.9700
Epoch 56/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0748 - accuracy: 0.9741
Epoch 57/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0807 - accuracy: 0.9726
Epoch 58/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0801 - accuracy: 0.9725
Epoch 59/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0688 - accuracy: 0.9762
Epoch 60/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0663 - accuracy: 0.9777
Epoch 61/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0803 - accuracy: 0.9724
Epoch 62/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0724 - accuracy: 0.9757
Epoch 63/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0770 - accuracy: 0.9739
Epoch 64/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0755 - accuracy: 0.9746
Epoch 65/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0641 - accuracy: 0.9780
Epoch 66/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0723 - accuracy: 0.9757
Epoch 67/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0690 - accuracy: 0.9760
Epoch 68/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0753 - accuracy: 0.9746
Epoch 69/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0756 - accuracy: 0.9746
Epoch 70/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0741 - accuracy: 0.9747
Epoch 71/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0781 - accuracy: 0.9774
Epoch 72/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0703 - accuracy: 0.9759
Epoch 73/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0776 - accuracy: 0.9735
Epoch 74/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0641 - accuracy: 0.9785
Epoch 75/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0700 - accuracy: 0.9762
Epoch 76/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0700 - accuracy: 0.9769
Epoch 77/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0713 - accuracy: 0.9764
Epoch 78/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0680 - accuracy: 0.9765
Epoch 79/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0759 - accuracy: 0.9767
Epoch 80/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0666 - accuracy: 0.9779
Epoch 81/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0682 - accuracy: 0.9770
Epoch 82/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0687 - accuracy: 0.9766
Epoch 83/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1275 - accuracy: 0.9755
Epoch 84/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0646 - accuracy: 0.9779
Epoch 85/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0813 - accuracy: 0.9729
Epoch 86/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0721 - accuracy: 0.9758
Epoch 87/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0656 - accuracy: 0.9777
Epoch 88/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0631 - accuracy: 0.9781
Epoch 89/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0713 - accuracy: 0.9761
Epoch 90/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0662 - accuracy: 0.9783
Epoch 91/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0674 - accuracy: 0.9772
Epoch 92/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0653 - accuracy: 0.9783
Epoch 93/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0637 - accuracy: 0.9783
Epoch 94/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0670 - accuracy: 0.9773
Epoch 95/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0623 - accuracy: 0.9783
Epoch 96/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0869 - accuracy: 0.9691
Epoch 97/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0632 - accuracy: 0.9793
Epoch 98/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0628 - accuracy: 0.9783
Epoch 99/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0659 - accuracy: 0.9773
Epoch 100/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0655 - accuracy: 0.9780
Epoch 101/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0612 - accuracy: 0.9795
Epoch 102/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0612 - accuracy: 0.9792
Epoch 103/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0624 - accuracy: 0.9785
Epoch 104/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0603 - accuracy: 0.9793
Epoch 105/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0598 - accuracy: 0.9798
Epoch 106/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0646 - accuracy: 0.9777
Epoch 107/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0650 - accuracy: 0.9777
Epoch 108/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0631 - accuracy: 0.9784
Epoch 109/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0635 - accuracy: 0.9783
Epoch 110/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0633 - accuracy: 0.9802
Epoch 111/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0614 - accuracy: 0.9793
Epoch 112/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0619 - accuracy: 0.9786
Epoch 113/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0603 - accuracy: 0.9794
Epoch 114/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0685 - accuracy: 0.9792
Epoch 115/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0600 - accuracy: 0.9793
Epoch 116/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0576 - accuracy: 0.9799
Epoch 117/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0602 - accuracy: 0.9792
Epoch 118/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0615 - accuracy: 0.9793
Epoch 119/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0568 - accuracy: 0.9810
Epoch 120/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0589 - accuracy: 0.9798
Epoch 121/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0599 - accuracy: 0.9790
Epoch 122/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0600 - accuracy: 0.9790
Epoch 123/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0585 - accuracy: 0.9799
Epoch 124/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0570 - accuracy: 0.9799
Epoch 125/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0683 - accuracy: 0.9784
Epoch 126/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0558 - accuracy: 0.9808
Epoch 127/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0610 - accuracy: 0.9789
Epoch 128/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0575 - accuracy: 0.9804
Epoch 129/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0575 - accuracy: 0.9800
Epoch 130/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0602 - accuracy: 0.9794
Epoch 131/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0929 - accuracy: 0.9807
Epoch 132/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0609 - accuracy: 0.9800
Epoch 133/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0597 - accuracy: 0.9793
Epoch 134/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0545 - accuracy: 0.9814
Epoch 135/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0591 - accuracy: 0.9801
Epoch 136/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0587 - accuracy: 0.9799
Epoch 137/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0573 - accuracy: 0.9805
Epoch 138/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0551 - accuracy: 0.9810
Epoch 139/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0598 - accuracy: 0.9790
Epoch 140/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0572 - accuracy: 0.9805
Epoch 141/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0616 - accuracy: 0.9787
Epoch 142/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0642 - accuracy: 0.9804
Epoch 143/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0704 - accuracy: 0.9809
Epoch 144/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0619 - accuracy: 0.9803
Epoch 145/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0558 - accuracy: 0.9803
Epoch 146/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0567 - accuracy: 0.9807
Epoch 147/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0559 - accuracy: 0.9806
Epoch 148/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0573 - accuracy: 0.9802
Epoch 149/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0702 - accuracy: 0.9808
Epoch 150/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0620 - accuracy: 0.9799

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_16_8_16_1_epochs", model=model, model_history=history)

Observations

The current neural network has four layers with 16, 8, 16 and 1 neurons in each layer. After being trained for 150 epochs, the model achieved an accuracy of 0.9799, it showed signs of overfitting or convergence. As a result, it may be beneficial to stick with 150 epochs and incerase the network complexity by adding more layers/neurons



Model Building with multiple layers and neurons (32-64-16-1) and 150 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(32, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(64, activation="relu"))
model.add(Dense(16, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=150, verbose=1)
Epoch 1/150
2798/2798 [==============================] - 4s 1ms/step - loss: 6.9746 - accuracy: 0.8281
Epoch 2/150
2798/2798 [==============================] - 3s 1ms/step - loss: 3.3738 - accuracy: 0.8790
Epoch 3/150
2798/2798 [==============================] - 3s 1ms/step - loss: 1.9047 - accuracy: 0.9007
Epoch 4/150
2798/2798 [==============================] - 3s 1ms/step - loss: 1.2352 - accuracy: 0.9095
Epoch 5/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6429 - accuracy: 0.9229
Epoch 6/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4353 - accuracy: 0.9295
Epoch 7/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2790 - accuracy: 0.9375
Epoch 8/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1857 - accuracy: 0.9449
Epoch 9/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1411 - accuracy: 0.9515
Epoch 10/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1472 - accuracy: 0.9550
Epoch 11/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1358 - accuracy: 0.9530
Epoch 12/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1057 - accuracy: 0.9626
Epoch 13/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1006 - accuracy: 0.9646
Epoch 14/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1077 - accuracy: 0.9651
Epoch 15/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0911 - accuracy: 0.9665
Epoch 16/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0826 - accuracy: 0.9683
Epoch 17/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0880 - accuracy: 0.9694
Epoch 18/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0769 - accuracy: 0.9707
Epoch 19/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0739 - accuracy: 0.9716
Epoch 20/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0752 - accuracy: 0.9711
Epoch 21/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0729 - accuracy: 0.9720
Epoch 22/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0674 - accuracy: 0.9737
Epoch 23/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0692 - accuracy: 0.9730
Epoch 24/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0657 - accuracy: 0.9740
Epoch 25/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0636 - accuracy: 0.9755
Epoch 26/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0636 - accuracy: 0.9748
Epoch 27/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0795 - accuracy: 0.9749
Epoch 28/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0641 - accuracy: 0.9744
Epoch 29/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0615 - accuracy: 0.9757
Epoch 30/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0619 - accuracy: 0.9758
Epoch 31/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0607 - accuracy: 0.9762
Epoch 32/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0666 - accuracy: 0.9750
Epoch 33/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0620 - accuracy: 0.9753
Epoch 34/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0555 - accuracy: 0.9782
Epoch 35/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0625 - accuracy: 0.9784
Epoch 36/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0629 - accuracy: 0.9800
Epoch 37/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0526 - accuracy: 0.9801
Epoch 38/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0479 - accuracy: 0.9816
Epoch 39/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0452 - accuracy: 0.9832
Epoch 40/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0630 - accuracy: 0.9777
Epoch 41/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0616 - accuracy: 0.9763
Epoch 42/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0558 - accuracy: 0.9775
Epoch 43/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0550 - accuracy: 0.9777
Epoch 44/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0533 - accuracy: 0.9783
Epoch 45/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0541 - accuracy: 0.9783
Epoch 46/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0566 - accuracy: 0.9778
Epoch 47/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0547 - accuracy: 0.9788
Epoch 48/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0572 - accuracy: 0.9789
Epoch 49/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0480 - accuracy: 0.9809
Epoch 50/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0519 - accuracy: 0.9796
Epoch 51/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0525 - accuracy: 0.9793
Epoch 52/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0565 - accuracy: 0.9791
Epoch 53/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0494 - accuracy: 0.9804
Epoch 54/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0504 - accuracy: 0.9804
Epoch 55/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0491 - accuracy: 0.9812
Epoch 56/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0486 - accuracy: 0.9808
Epoch 57/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0460 - accuracy: 0.9823
Epoch 58/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0446 - accuracy: 0.9826
Epoch 59/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0465 - accuracy: 0.9818
Epoch 60/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0705 - accuracy: 0.9814
Epoch 61/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0442 - accuracy: 0.9822
Epoch 62/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0433 - accuracy: 0.9826
Epoch 63/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0449 - accuracy: 0.9827
Epoch 64/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0416 - accuracy: 0.9834
Epoch 65/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0435 - accuracy: 0.9833
Epoch 66/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0429 - accuracy: 0.9831
Epoch 67/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0439 - accuracy: 0.9830
Epoch 68/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0741 - accuracy: 0.9836
Epoch 69/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0419 - accuracy: 0.9834
Epoch 70/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0424 - accuracy: 0.9831
Epoch 71/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0469 - accuracy: 0.9824
Epoch 72/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0586 - accuracy: 0.9837
Epoch 73/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0444 - accuracy: 0.9830
Epoch 74/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0427 - accuracy: 0.9833
Epoch 75/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0460 - accuracy: 0.9832
Epoch 76/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0447 - accuracy: 0.9828
Epoch 77/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0413 - accuracy: 0.9834
Epoch 78/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0397 - accuracy: 0.9837
Epoch 79/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0428 - accuracy: 0.9839
Epoch 80/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0414 - accuracy: 0.9846
Epoch 81/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0421 - accuracy: 0.9839
Epoch 82/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0394 - accuracy: 0.9841
Epoch 83/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0413 - accuracy: 0.9841
Epoch 84/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0418 - accuracy: 0.9843
Epoch 85/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0420 - accuracy: 0.9835
Epoch 86/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0402 - accuracy: 0.9840
Epoch 87/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0397 - accuracy: 0.9842
Epoch 88/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0380 - accuracy: 0.9851
Epoch 89/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0404 - accuracy: 0.9842
Epoch 90/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0394 - accuracy: 0.9844
Epoch 91/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0400 - accuracy: 0.9843
Epoch 92/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0380 - accuracy: 0.9846
Epoch 93/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0419 - accuracy: 0.9847
Epoch 94/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0420 - accuracy: 0.9842
Epoch 95/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0457 - accuracy: 0.9823
Epoch 96/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0412 - accuracy: 0.9840
Epoch 97/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0440 - accuracy: 0.9833
Epoch 98/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0406 - accuracy: 0.9837
Epoch 99/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0421 - accuracy: 0.9839
Epoch 100/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0451 - accuracy: 0.9832
Epoch 101/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0502 - accuracy: 0.9851
Epoch 102/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0426 - accuracy: 0.9830
Epoch 103/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0433 - accuracy: 0.9836
Epoch 104/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0404 - accuracy: 0.9838
Epoch 105/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0420 - accuracy: 0.9840
Epoch 106/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0383 - accuracy: 0.9846
Epoch 107/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0420 - accuracy: 0.9845
Epoch 108/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0372 - accuracy: 0.9855
Epoch 109/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0399 - accuracy: 0.9851
Epoch 110/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0388 - accuracy: 0.9849
Epoch 111/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0378 - accuracy: 0.9853
Epoch 112/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0390 - accuracy: 0.9847
Epoch 113/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0462 - accuracy: 0.9831
Epoch 114/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0358 - accuracy: 0.9853
Epoch 115/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0364 - accuracy: 0.9855
Epoch 116/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0441 - accuracy: 0.9849
Epoch 117/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0386 - accuracy: 0.9857
Epoch 118/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0367 - accuracy: 0.9856
Epoch 119/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0363 - accuracy: 0.9854
Epoch 120/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0377 - accuracy: 0.9853
Epoch 121/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0352 - accuracy: 0.9860
Epoch 122/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0533 - accuracy: 0.9833
Epoch 123/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0427 - accuracy: 0.9848
Epoch 124/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0378 - accuracy: 0.9854
Epoch 125/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0401 - accuracy: 0.9845
Epoch 126/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0358 - accuracy: 0.9857
Epoch 127/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0394 - accuracy: 0.9859
Epoch 128/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0357 - accuracy: 0.9869
Epoch 129/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0369 - accuracy: 0.9865
Epoch 130/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0359 - accuracy: 0.9867
Epoch 131/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3381 - accuracy: 0.9862
Epoch 132/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0531 - accuracy: 0.9869
Epoch 133/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0324 - accuracy: 0.9883
Epoch 134/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0341 - accuracy: 0.9877
Epoch 135/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0312 - accuracy: 0.9878
Epoch 136/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0336 - accuracy: 0.9872
Epoch 137/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0312 - accuracy: 0.9877
Epoch 138/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0323 - accuracy: 0.9879
Epoch 139/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0323 - accuracy: 0.9882
Epoch 140/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0326 - accuracy: 0.9875
Epoch 141/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0315 - accuracy: 0.9882
Epoch 142/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0303 - accuracy: 0.9883
Epoch 143/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0320 - accuracy: 0.9878
Epoch 144/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0343 - accuracy: 0.9882
Epoch 145/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0328 - accuracy: 0.9878
Epoch 146/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0308 - accuracy: 0.9882
Epoch 147/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0293 - accuracy: 0.9889
Epoch 148/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0308 - accuracy: 0.9881
Epoch 149/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0288 - accuracy: 0.9886
Epoch 150/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0311 - accuracy: 0.9881

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_32_64_16_1_150_epochs", model=model, model_history=history)

Observations

The current neural network has four layers with 32, 64, 16 and 1 neurons in each layer. After being trained for 150 epochs, the model achieved an accuracy of 0.9881, it showed signs of overfitting or convergence. As a result, it may be beneficial to decrease the number of epochs and incerase the network complexity by adding more layers/neurons to achieve 100 percent accuracy



Model Building with multiple layers and neurons (64-32-16-1) and 150 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(64, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(32, activation="relu"))
model.add(Dense(16, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=150, verbose=1)
Epoch 1/150
2798/2798 [==============================] - 4s 1ms/step - loss: 8.3898 - accuracy: 0.8534
Epoch 2/150
2798/2798 [==============================] - 3s 1ms/step - loss: 2.2237 - accuracy: 0.8921
Epoch 3/150
2798/2798 [==============================] - 3s 1ms/step - loss: 1.0122 - accuracy: 0.9037
Epoch 4/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2735 - accuracy: 0.9100
Epoch 5/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2444 - accuracy: 0.9061
Epoch 6/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2401 - accuracy: 0.9184
Epoch 7/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1902 - accuracy: 0.9323
Epoch 8/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2376 - accuracy: 0.9304
Epoch 9/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1615 - accuracy: 0.9401
Epoch 10/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1812 - accuracy: 0.9465
Epoch 11/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1213 - accuracy: 0.9544
Epoch 12/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1116 - accuracy: 0.9576
Epoch 13/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1097 - accuracy: 0.9591
Epoch 14/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2718 - accuracy: 0.9610
Epoch 15/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1066 - accuracy: 0.9612
Epoch 16/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1107 - accuracy: 0.9601
Epoch 17/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1142 - accuracy: 0.9593
Epoch 18/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1036 - accuracy: 0.9623
Epoch 19/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1208 - accuracy: 0.9560
Epoch 20/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1026 - accuracy: 0.9629
Epoch 21/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1026 - accuracy: 0.9630
Epoch 22/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1045 - accuracy: 0.9642
Epoch 23/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3263 - accuracy: 0.8573
Epoch 24/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3228 - accuracy: 0.8611
Epoch 25/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3269 - accuracy: 0.8593
Epoch 26/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2924 - accuracy: 0.8782
Epoch 27/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1001 - accuracy: 0.9670
Epoch 28/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0839 - accuracy: 0.9700
Epoch 29/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1144 - accuracy: 0.9689
Epoch 30/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0823 - accuracy: 0.9717
Epoch 31/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0852 - accuracy: 0.9690
Epoch 32/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0862 - accuracy: 0.9680
Epoch 33/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0778 - accuracy: 0.9710
Epoch 34/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0789 - accuracy: 0.9715
Epoch 35/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0845 - accuracy: 0.9694
Epoch 36/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0776 - accuracy: 0.9715
Epoch 37/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0883 - accuracy: 0.9695
Epoch 38/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0858 - accuracy: 0.9697
Epoch 39/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0744 - accuracy: 0.9730
Epoch 40/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1096 - accuracy: 0.9740
Epoch 41/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0773 - accuracy: 0.9724
Epoch 42/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0747 - accuracy: 0.9736
Epoch 43/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0950 - accuracy: 0.9668
Epoch 44/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0761 - accuracy: 0.9733
Epoch 45/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0789 - accuracy: 0.9728
Epoch 46/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0731 - accuracy: 0.9743
Epoch 47/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2030 - accuracy: 0.9195
Epoch 48/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1905 - accuracy: 0.9393
Epoch 49/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1236 - accuracy: 0.9506
Epoch 50/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1023 - accuracy: 0.9590
Epoch 51/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1260 - accuracy: 0.9620
Epoch 52/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0969 - accuracy: 0.9632
Epoch 53/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0831 - accuracy: 0.9681
Epoch 54/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0834 - accuracy: 0.9707
Epoch 55/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0774 - accuracy: 0.9737
Epoch 56/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0758 - accuracy: 0.9752
Epoch 57/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0759 - accuracy: 0.9747
Epoch 58/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0727 - accuracy: 0.9760
Epoch 59/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0753 - accuracy: 0.9749
Epoch 60/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0869 - accuracy: 0.9768
Epoch 61/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0733 - accuracy: 0.9759
Epoch 62/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0661 - accuracy: 0.9776
Epoch 63/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0890 - accuracy: 0.9773
Epoch 64/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0661 - accuracy: 0.9779
Epoch 65/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0701 - accuracy: 0.9769
Epoch 66/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0647 - accuracy: 0.9783
Epoch 67/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0651 - accuracy: 0.9783
Epoch 68/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0713 - accuracy: 0.9769
Epoch 69/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0646 - accuracy: 0.9781
Epoch 70/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0696 - accuracy: 0.9771
Epoch 71/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0667 - accuracy: 0.9774
Epoch 72/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0635 - accuracy: 0.9792
Epoch 73/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0976 - accuracy: 0.9770
Epoch 74/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1093 - accuracy: 0.9744
Epoch 75/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0682 - accuracy: 0.9777
Epoch 76/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0670 - accuracy: 0.9787
Epoch 77/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0705 - accuracy: 0.9765
Epoch 78/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0694 - accuracy: 0.9764
Epoch 79/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0637 - accuracy: 0.9793
Epoch 80/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0749 - accuracy: 0.9749
Epoch 81/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0715 - accuracy: 0.9764
Epoch 82/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0647 - accuracy: 0.9785
Epoch 83/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1079 - accuracy: 0.9644
Epoch 84/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0780 - accuracy: 0.9743
Epoch 85/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0724 - accuracy: 0.9757
Epoch 86/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0660 - accuracy: 0.9796
Epoch 87/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0837 - accuracy: 0.9789
Epoch 88/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0688 - accuracy: 0.9768
Epoch 89/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0724 - accuracy: 0.9778
Epoch 90/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0629 - accuracy: 0.9788
Epoch 91/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0589 - accuracy: 0.9800
Epoch 92/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0588 - accuracy: 0.9814
Epoch 93/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0653 - accuracy: 0.9790
Epoch 94/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0579 - accuracy: 0.9798
Epoch 95/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0618 - accuracy: 0.9790
Epoch 96/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1616 - accuracy: 0.9827
Epoch 97/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0555 - accuracy: 0.9815
Epoch 98/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0533 - accuracy: 0.9818
Epoch 99/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0546 - accuracy: 0.9818
Epoch 100/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0586 - accuracy: 0.9803
Epoch 101/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0541 - accuracy: 0.9825
Epoch 102/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0586 - accuracy: 0.9808
Epoch 103/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0529 - accuracy: 0.9828
Epoch 104/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0994 - accuracy: 0.9824
Epoch 105/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0663 - accuracy: 0.9832
Epoch 106/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0539 - accuracy: 0.9814
Epoch 107/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0509 - accuracy: 0.9827
Epoch 108/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0515 - accuracy: 0.9831
Epoch 109/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0525 - accuracy: 0.9832
Epoch 110/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0488 - accuracy: 0.9839
Epoch 111/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0528 - accuracy: 0.9823
Epoch 112/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0567 - accuracy: 0.9821
Epoch 113/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0606 - accuracy: 0.9806
Epoch 114/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0466 - accuracy: 0.9848
Epoch 115/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0509 - accuracy: 0.9834
Epoch 116/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0541 - accuracy: 0.9821
Epoch 117/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0449 - accuracy: 0.9851
Epoch 118/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0477 - accuracy: 0.9848
Epoch 119/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0537 - accuracy: 0.9827
Epoch 120/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0523 - accuracy: 0.9831
Epoch 121/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0505 - accuracy: 0.9835
Epoch 122/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0495 - accuracy: 0.9841
Epoch 123/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0495 - accuracy: 0.9835
Epoch 124/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0474 - accuracy: 0.9848
Epoch 125/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0730 - accuracy: 0.9762
Epoch 126/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0488 - accuracy: 0.9844
Epoch 127/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0505 - accuracy: 0.9833
Epoch 128/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0549 - accuracy: 0.9817
Epoch 129/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0591 - accuracy: 0.9804
Epoch 130/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0542 - accuracy: 0.9824
Epoch 131/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0545 - accuracy: 0.9825
Epoch 132/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0474 - accuracy: 0.9846
Epoch 133/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0435 - accuracy: 0.9857
Epoch 134/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0510 - accuracy: 0.9829
Epoch 135/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0528 - accuracy: 0.9825
Epoch 136/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0510 - accuracy: 0.9834
Epoch 137/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0507 - accuracy: 0.9833
Epoch 138/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0524 - accuracy: 0.9822
Epoch 139/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1332 - accuracy: 0.9848
Epoch 140/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0543 - accuracy: 0.9830
Epoch 141/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0511 - accuracy: 0.9835
Epoch 142/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0509 - accuracy: 0.9829
Epoch 143/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0501 - accuracy: 0.9830
Epoch 144/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0520 - accuracy: 0.9834
Epoch 145/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0445 - accuracy: 0.9856
Epoch 146/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0513 - accuracy: 0.9830
Epoch 147/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0487 - accuracy: 0.9839
Epoch 148/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0527 - accuracy: 0.9826
Epoch 149/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0500 - accuracy: 0.9830
Epoch 150/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0499 - accuracy: 0.9844

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_64_32_16_1_150_epochs", model=model, model_history=history)

Observations

The current neural network has four layers with 64, 32, 16 and 1 neurons in each layer. After being trained for 150 epochs, the model achieved an accuracy of 0.9844, it showed signs of overfitting or convergence. As a result, it may be beneficial to stick with 150 epochs and incerase the network complexity by adding more layers/neurons to achieve 100 percent accuracy.



Model Building with multiple layers and neurons (128-64-64-1) and 150 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(128, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(64, activation="relu"))
model.add(Dense(64, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=150, verbose=1)
Epoch 1/150
2798/2798 [==============================] - 4s 1ms/step - loss: 4.6028 - accuracy: 0.8396
Epoch 2/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5966 - accuracy: 0.8889
Epoch 3/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2946 - accuracy: 0.8983
Epoch 4/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2762 - accuracy: 0.8996
Epoch 5/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2464 - accuracy: 0.9156
Epoch 6/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3112 - accuracy: 0.8846
Epoch 7/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2563 - accuracy: 0.9142
Epoch 8/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4542 - accuracy: 0.9061
Epoch 9/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5192 - accuracy: 0.6830
Epoch 10/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6505 - accuracy: 0.5576
Epoch 11/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6487 - accuracy: 0.5606
Epoch 12/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.8574 - accuracy: 0.5575
Epoch 13/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6578 - accuracy: 0.5552
Epoch 14/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6481 - accuracy: 0.5614
Epoch 15/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6595 - accuracy: 0.5591
Epoch 16/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6490 - accuracy: 0.5613
Epoch 17/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6532 - accuracy: 0.5537
Epoch 18/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6533 - accuracy: 0.5535
Epoch 19/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6533 - accuracy: 0.5539
Epoch 20/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6533 - accuracy: 0.5539
Epoch 21/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6532 - accuracy: 0.5537
Epoch 22/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6668 - accuracy: 0.5549
Epoch 23/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6481 - accuracy: 0.5599
Epoch 24/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6482 - accuracy: 0.5597
Epoch 25/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6482 - accuracy: 0.5599
Epoch 26/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6482 - accuracy: 0.5599
Epoch 27/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6482 - accuracy: 0.5599
Epoch 28/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6730 - accuracy: 0.5590
Epoch 29/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6905 - accuracy: 0.5613
Epoch 30/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6525 - accuracy: 0.5607
Epoch 31/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5599
Epoch 32/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6519 - accuracy: 0.5622
Epoch 33/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6500 - accuracy: 0.5596
Epoch 34/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6468 - accuracy: 0.5620
Epoch 35/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6551 - accuracy: 0.5619
Epoch 36/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6441 - accuracy: 0.5681
Epoch 37/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6496 - accuracy: 0.5619
Epoch 38/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6651 - accuracy: 0.5653
Epoch 39/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6446 - accuracy: 0.5657
Epoch 40/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6421 - accuracy: 0.5696
Epoch 41/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 42/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 43/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 44/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 45/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 46/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 47/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 48/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 49/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 50/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 51/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 52/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 53/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 54/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 55/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 56/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 57/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 58/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 59/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 60/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 61/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 62/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 63/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 64/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 65/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 66/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 67/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 68/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 69/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 70/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 71/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 72/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 73/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 74/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 75/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 76/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 77/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 78/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 79/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 80/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 81/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 82/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 83/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 84/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 85/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 86/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6476 - accuracy: 0.5610
Epoch 87/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 88/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 89/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 90/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 91/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 92/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 93/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 94/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 95/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 96/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 97/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 98/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 99/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 100/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 101/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 102/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 103/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 104/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 105/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 106/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 107/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 108/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 109/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 110/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 111/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 112/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 113/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 114/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 115/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 116/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 117/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 118/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 119/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 120/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 121/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 122/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 123/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 124/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 125/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 126/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 127/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 128/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 129/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 130/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 131/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 132/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 133/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 134/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 135/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 136/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 137/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 138/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 139/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 140/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 141/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 142/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 143/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 144/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 145/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 146/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 147/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 148/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 149/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610
Epoch 150/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.6475 - accuracy: 0.5610

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_128_64_64_1_150_epochs", model=model, model_history=history)

Observations

The accuracy of the model increases rapidly during the initial training phase with larger increments, but after 9 epochs, it decreases significantly. This may be due to the unbalanced number of neurons in the layers. Specifically, the first layer has a large number of neurons, while the second and third layers have relatively fewer neurons. To improve the model's performance, one approach is to increase the number of neurons in the third layer or decrease the number of neurons in the first layer.



Model Building with multiple layers and neurons (128-128-64-1) and 150 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(128, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(64, activation="relu"))
model.add(Dense(32, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=150, verbose=1)
Epoch 1/150
2798/2798 [==============================] - 4s 1ms/step - loss: 10.3918 - accuracy: 0.8388
Epoch 2/150
2798/2798 [==============================] - 3s 1ms/step - loss: 2.8516 - accuracy: 0.8853
Epoch 3/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.9283 - accuracy: 0.9006
Epoch 4/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2398 - accuracy: 0.9238
Epoch 5/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2639 - accuracy: 0.9057
Epoch 6/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3715 - accuracy: 0.8338
Epoch 7/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.5120 - accuracy: 0.9220
Epoch 8/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2062 - accuracy: 0.9261
Epoch 9/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1977 - accuracy: 0.9276
Epoch 10/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1943 - accuracy: 0.9284
Epoch 11/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1983 - accuracy: 0.9261
Epoch 12/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1854 - accuracy: 0.9309
Epoch 13/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2497 - accuracy: 0.9040
Epoch 14/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1602 - accuracy: 0.9424
Epoch 15/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1405 - accuracy: 0.9508
Epoch 16/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1216 - accuracy: 0.9574
Epoch 17/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1686 - accuracy: 0.9433
Epoch 18/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1279 - accuracy: 0.9549
Epoch 19/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1164 - accuracy: 0.9589
Epoch 20/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1708 - accuracy: 0.9628
Epoch 21/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1077 - accuracy: 0.9628
Epoch 22/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1285 - accuracy: 0.9565
Epoch 23/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1098 - accuracy: 0.9624
Epoch 24/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1136 - accuracy: 0.9600
Epoch 25/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1098 - accuracy: 0.9645
Epoch 26/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1097 - accuracy: 0.9641
Epoch 27/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1167 - accuracy: 0.9631
Epoch 28/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0963 - accuracy: 0.9672
Epoch 29/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0965 - accuracy: 0.9676
Epoch 30/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1081 - accuracy: 0.9644
Epoch 31/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0762 - accuracy: 0.9740
Epoch 32/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0869 - accuracy: 0.9707
Epoch 33/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0979 - accuracy: 0.9678
Epoch 34/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0789 - accuracy: 0.9736
Epoch 35/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1584 - accuracy: 0.9727
Epoch 36/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1652 - accuracy: 0.9555
Epoch 37/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0837 - accuracy: 0.9723
Epoch 38/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0838 - accuracy: 0.9714
Epoch 39/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0857 - accuracy: 0.9703
Epoch 40/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0714 - accuracy: 0.9759
Epoch 41/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0796 - accuracy: 0.9729
Epoch 42/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0832 - accuracy: 0.9723
Epoch 43/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0712 - accuracy: 0.9759
Epoch 44/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0740 - accuracy: 0.9758
Epoch 45/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0746 - accuracy: 0.9755
Epoch 46/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0833 - accuracy: 0.9729
Epoch 47/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0885 - accuracy: 0.9708
Epoch 48/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0751 - accuracy: 0.9754
Epoch 49/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0866 - accuracy: 0.9720
Epoch 50/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0710 - accuracy: 0.9765
Epoch 51/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0864 - accuracy: 0.9729
Epoch 52/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0971 - accuracy: 0.9676
Epoch 53/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0722 - accuracy: 0.9756
Epoch 54/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1777 - accuracy: 0.9682
Epoch 55/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0761 - accuracy: 0.9745
Epoch 56/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0862 - accuracy: 0.9712
Epoch 57/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0676 - accuracy: 0.9770
Epoch 58/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0870 - accuracy: 0.9722
Epoch 59/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0827 - accuracy: 0.9725
Epoch 60/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0815 - accuracy: 0.9735
Epoch 61/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0882 - accuracy: 0.9709
Epoch 62/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1278 - accuracy: 0.9584
Epoch 63/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2114 - accuracy: 0.9232
Epoch 64/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1593 - accuracy: 0.9409
Epoch 65/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1418 - accuracy: 0.9464
Epoch 66/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1328 - accuracy: 0.9494
Epoch 67/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1268 - accuracy: 0.9501
Epoch 68/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1244 - accuracy: 0.9527
Epoch 69/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1123 - accuracy: 0.9561
Epoch 70/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1312 - accuracy: 0.9549
Epoch 71/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1794 - accuracy: 0.9549
Epoch 72/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1186 - accuracy: 0.9549
Epoch 73/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1091 - accuracy: 0.9572
Epoch 74/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1079 - accuracy: 0.9581
Epoch 75/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1016 - accuracy: 0.9613
Epoch 76/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1003 - accuracy: 0.9625
Epoch 77/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1007 - accuracy: 0.9631
Epoch 78/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0875 - accuracy: 0.9670
Epoch 79/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0831 - accuracy: 0.9683
Epoch 80/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4008 - accuracy: 0.9671
Epoch 81/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1619 - accuracy: 0.9719
Epoch 82/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0739 - accuracy: 0.9749
Epoch 83/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0809 - accuracy: 0.9726
Epoch 84/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2726 - accuracy: 0.9759
Epoch 85/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1074 - accuracy: 0.9759
Epoch 86/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0719 - accuracy: 0.9756
Epoch 87/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0798 - accuracy: 0.9734
Epoch 88/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0602 - accuracy: 0.9798
Epoch 89/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0889 - accuracy: 0.9766
Epoch 90/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0780 - accuracy: 0.9726
Epoch 91/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0746 - accuracy: 0.9754
Epoch 92/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0696 - accuracy: 0.9766
Epoch 93/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0623 - accuracy: 0.9801
Epoch 94/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0535 - accuracy: 0.9829
Epoch 95/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0549 - accuracy: 0.9820
Epoch 96/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0726 - accuracy: 0.9779
Epoch 97/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1391 - accuracy: 0.9792
Epoch 98/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1993 - accuracy: 0.9812
Epoch 99/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0565 - accuracy: 0.9821
Epoch 100/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0740 - accuracy: 0.9758
Epoch 101/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0607 - accuracy: 0.9810
Epoch 102/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0508 - accuracy: 0.9835
Epoch 103/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0682 - accuracy: 0.9768
Epoch 104/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0615 - accuracy: 0.9784
Epoch 105/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0663 - accuracy: 0.9794
Epoch 106/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0606 - accuracy: 0.9797
Epoch 107/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0559 - accuracy: 0.9811
Epoch 108/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0589 - accuracy: 0.9806
Epoch 109/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0599 - accuracy: 0.9796
Epoch 110/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0670 - accuracy: 0.9771
Epoch 111/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0569 - accuracy: 0.9809
Epoch 112/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2414 - accuracy: 0.9781
Epoch 113/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0555 - accuracy: 0.9822
Epoch 114/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0521 - accuracy: 0.9833
Epoch 115/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0631 - accuracy: 0.9772
Epoch 116/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0705 - accuracy: 0.9764
Epoch 117/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0628 - accuracy: 0.9774
Epoch 118/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0714 - accuracy: 0.9766
Epoch 119/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0595 - accuracy: 0.9809
Epoch 120/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0652 - accuracy: 0.9789
Epoch 121/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3057 - accuracy: 0.9808
Epoch 122/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0804 - accuracy: 0.9714
Epoch 123/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0774 - accuracy: 0.9779
Epoch 124/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0593 - accuracy: 0.9793
Epoch 125/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0591 - accuracy: 0.9787
Epoch 126/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0602 - accuracy: 0.9795
Epoch 127/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0577 - accuracy: 0.9804
Epoch 128/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0539 - accuracy: 0.9817
Epoch 129/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1302 - accuracy: 0.9792
Epoch 130/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.3901 - accuracy: 0.9772
Epoch 131/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0642 - accuracy: 0.9773
Epoch 132/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0661 - accuracy: 0.9768
Epoch 133/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0595 - accuracy: 0.9788
Epoch 134/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0676 - accuracy: 0.9762
Epoch 135/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0620 - accuracy: 0.9773
Epoch 136/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0606 - accuracy: 0.9776
Epoch 137/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0615 - accuracy: 0.9778
Epoch 138/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0612 - accuracy: 0.9776
Epoch 139/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0588 - accuracy: 0.9789
Epoch 140/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0868 - accuracy: 0.9782
Epoch 141/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0715 - accuracy: 0.9770
Epoch 142/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0590 - accuracy: 0.9784
Epoch 143/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0589 - accuracy: 0.9783
Epoch 144/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0585 - accuracy: 0.9786
Epoch 145/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0608 - accuracy: 0.9774
Epoch 146/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0583 - accuracy: 0.9784
Epoch 147/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0588 - accuracy: 0.9786
Epoch 148/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0602 - accuracy: 0.9785
Epoch 149/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0611 - accuracy: 0.9783
Epoch 150/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0590 - accuracy: 0.9790

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_128_128_64_1_150_epochs", model=model, model_history=history)

Observations

The current neural network has four layers with 128, 128, 64 and 1 neurons in each layer. After being trained for 150 epochs, the model achieved an accuracy of 0.9790, it showed signs of overfitting or convergence. As a result, it may be beneficial to stick with 150 epochs and incerase the network complexity by adding more layers/neurons to achieve 100 percent accuracy.



Model Building with multiple layers and neurons (16-32-32-16-1) and 150 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(16, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(32, activation="relu"))
model.add(Dense(32, activation="relu"))
model.add(Dense(16, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=150, verbose=1)
Epoch 1/150
2798/2798 [==============================] - 4s 1ms/step - loss: 2.8241 - accuracy: 0.8362
Epoch 2/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.9021 - accuracy: 0.8768
Epoch 3/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.4592 - accuracy: 0.8905
Epoch 4/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2601 - accuracy: 0.9107
Epoch 5/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2349 - accuracy: 0.9186
Epoch 6/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2241 - accuracy: 0.9250
Epoch 7/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2119 - accuracy: 0.9184
Epoch 8/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2357 - accuracy: 0.9195
Epoch 9/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1849 - accuracy: 0.9242
Epoch 10/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1497 - accuracy: 0.9430
Epoch 11/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.2489 - accuracy: 0.9436
Epoch 12/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1716 - accuracy: 0.9334
Epoch 13/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1449 - accuracy: 0.9463
Epoch 14/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1239 - accuracy: 0.9562
Epoch 15/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1154 - accuracy: 0.9597
Epoch 16/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1200 - accuracy: 0.9576
Epoch 17/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1028 - accuracy: 0.9636
Epoch 18/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1123 - accuracy: 0.9656
Epoch 19/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0944 - accuracy: 0.9666
Epoch 20/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0883 - accuracy: 0.9686
Epoch 21/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0855 - accuracy: 0.9692
Epoch 22/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0837 - accuracy: 0.9704
Epoch 23/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0818 - accuracy: 0.9707
Epoch 24/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0780 - accuracy: 0.9716
Epoch 25/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0784 - accuracy: 0.9709
Epoch 26/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0742 - accuracy: 0.9722
Epoch 27/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0727 - accuracy: 0.9722
Epoch 28/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0749 - accuracy: 0.9728
Epoch 29/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0693 - accuracy: 0.9743
Epoch 30/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0678 - accuracy: 0.9750
Epoch 31/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0649 - accuracy: 0.9757
Epoch 32/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0672 - accuracy: 0.9749
Epoch 33/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0652 - accuracy: 0.9754
Epoch 34/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0642 - accuracy: 0.9766
Epoch 35/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0621 - accuracy: 0.9760
Epoch 36/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0575 - accuracy: 0.9780
Epoch 37/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0557 - accuracy: 0.9791
Epoch 38/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0561 - accuracy: 0.9795
Epoch 39/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0554 - accuracy: 0.9799
Epoch 40/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0522 - accuracy: 0.9810
Epoch 41/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0540 - accuracy: 0.9796
Epoch 42/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0628 - accuracy: 0.9796
Epoch 43/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0565 - accuracy: 0.9794
Epoch 44/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0493 - accuracy: 0.9815
Epoch 45/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0510 - accuracy: 0.9811
Epoch 46/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0492 - accuracy: 0.9814
Epoch 47/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0495 - accuracy: 0.9814
Epoch 48/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0583 - accuracy: 0.9803
Epoch 49/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0481 - accuracy: 0.9815
Epoch 50/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0471 - accuracy: 0.9819
Epoch 51/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0507 - accuracy: 0.9817
Epoch 52/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0460 - accuracy: 0.9822
Epoch 53/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0486 - accuracy: 0.9817
Epoch 54/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0487 - accuracy: 0.9814
Epoch 55/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0451 - accuracy: 0.9831
Epoch 56/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0473 - accuracy: 0.9816
Epoch 57/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0445 - accuracy: 0.9829
Epoch 58/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0673 - accuracy: 0.9816
Epoch 59/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0460 - accuracy: 0.9827
Epoch 60/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0453 - accuracy: 0.9825
Epoch 61/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0441 - accuracy: 0.9829
Epoch 62/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0424 - accuracy: 0.9833
Epoch 63/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0436 - accuracy: 0.9835
Epoch 64/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0452 - accuracy: 0.9834
Epoch 65/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0425 - accuracy: 0.9838
Epoch 66/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0432 - accuracy: 0.9836
Epoch 67/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0444 - accuracy: 0.9831
Epoch 68/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0505 - accuracy: 0.9822
Epoch 69/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0484 - accuracy: 0.9831
Epoch 70/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0454 - accuracy: 0.9829
Epoch 71/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0468 - accuracy: 0.9831
Epoch 72/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0446 - accuracy: 0.9831
Epoch 73/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0462 - accuracy: 0.9833
Epoch 74/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0432 - accuracy: 0.9839
Epoch 75/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0450 - accuracy: 0.9838
Epoch 76/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0457 - accuracy: 0.9829
Epoch 77/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0429 - accuracy: 0.9843
Epoch 78/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0581 - accuracy: 0.9819
Epoch 79/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0420 - accuracy: 0.9840
Epoch 80/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0413 - accuracy: 0.9841
Epoch 81/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0430 - accuracy: 0.9837
Epoch 82/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0472 - accuracy: 0.9834
Epoch 83/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0471 - accuracy: 0.9836
Epoch 84/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0383 - accuracy: 0.9853
Epoch 85/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0477 - accuracy: 0.9836
Epoch 86/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0406 - accuracy: 0.9847
Epoch 87/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0419 - accuracy: 0.9843
Epoch 88/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0388 - accuracy: 0.9849
Epoch 89/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0562 - accuracy: 0.9837
Epoch 90/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0434 - accuracy: 0.9843
Epoch 91/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0413 - accuracy: 0.9845
Epoch 92/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0466 - accuracy: 0.9841
Epoch 93/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0418 - accuracy: 0.9841
Epoch 94/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0401 - accuracy: 0.9844
Epoch 95/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0384 - accuracy: 0.9854
Epoch 96/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0433 - accuracy: 0.9842
Epoch 97/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0480 - accuracy: 0.9851
Epoch 98/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0381 - accuracy: 0.9852
Epoch 99/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0439 - accuracy: 0.9837
Epoch 100/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0404 - accuracy: 0.9845
Epoch 101/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0414 - accuracy: 0.9838
Epoch 102/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0491 - accuracy: 0.9839
Epoch 103/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0423 - accuracy: 0.9858
Epoch 104/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0371 - accuracy: 0.9856
Epoch 105/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0389 - accuracy: 0.9848
Epoch 106/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0394 - accuracy: 0.9854
Epoch 107/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0390 - accuracy: 0.9855
Epoch 108/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0423 - accuracy: 0.9843
Epoch 109/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0378 - accuracy: 0.9852
Epoch 110/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0545 - accuracy: 0.9827
Epoch 111/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0386 - accuracy: 0.9856
Epoch 112/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0362 - accuracy: 0.9861
Epoch 113/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1415 - accuracy: 0.9472
Epoch 114/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1557 - accuracy: 0.9393
Epoch 115/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0592 - accuracy: 0.9800
Epoch 116/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0476 - accuracy: 0.9834
Epoch 117/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0474 - accuracy: 0.9830
Epoch 118/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0450 - accuracy: 0.9843
Epoch 119/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0448 - accuracy: 0.9844
Epoch 120/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0436 - accuracy: 0.9845
Epoch 121/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0406 - accuracy: 0.9855
Epoch 122/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0904 - accuracy: 0.9811
Epoch 123/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0579 - accuracy: 0.9855
Epoch 124/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0472 - accuracy: 0.9836
Epoch 125/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0389 - accuracy: 0.9861
Epoch 126/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0431 - accuracy: 0.9850
Epoch 127/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0403 - accuracy: 0.9856
Epoch 128/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0404 - accuracy: 0.9858
Epoch 129/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0402 - accuracy: 0.9855
Epoch 130/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0381 - accuracy: 0.9866
Epoch 131/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0385 - accuracy: 0.9858
Epoch 132/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0455 - accuracy: 0.9845
Epoch 133/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.1020 - accuracy: 0.9832
Epoch 134/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0357 - accuracy: 0.9872
Epoch 135/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0384 - accuracy: 0.9863
Epoch 136/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0404 - accuracy: 0.9859
Epoch 137/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0475 - accuracy: 0.9845
Epoch 138/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0488 - accuracy: 0.9868
Epoch 139/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0420 - accuracy: 0.9850
Epoch 140/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0350 - accuracy: 0.9870
Epoch 141/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0381 - accuracy: 0.9864
Epoch 142/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0368 - accuracy: 0.9868
Epoch 143/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0589 - accuracy: 0.9797
Epoch 144/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0516 - accuracy: 0.9848
Epoch 145/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0455 - accuracy: 0.9841
Epoch 146/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0408 - accuracy: 0.9850
Epoch 147/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0390 - accuracy: 0.9859
Epoch 148/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0544 - accuracy: 0.9824
Epoch 149/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0377 - accuracy: 0.9859
Epoch 150/150
2798/2798 [==============================] - 3s 1ms/step - loss: 0.0379 - accuracy: 0.9861

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_16_32_32_16_1_150_epochs", model=model, model_history=history)

Observations

The current neural network has four layers with 16, 32, 32, 16 and 1 neurons in each layer. After being trained for 150 epochs, the model achieved an accuracy of 0.9861, it showed signs of overfitting or convergence. As a result, it may be beneficial to stick with 150 epochs and incerase the network complexity by adding more layers/neurons to achieve 100 percent accuracy.



Model Building with multiple layers and neurons (32-128-64-32-1) and 150 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(32, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(128, activation="relu"))
model.add(Dense(64, activation="relu"))
model.add(Dense(32, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=150, verbose=1)
Epoch 1/150
2798/2798 [==============================] - 4s 1ms/step - loss: 6.9395 - accuracy: 0.8279
Epoch 2/150
2798/2798 [==============================] - 4s 1ms/step - loss: 1.0918 - accuracy: 0.8838
Epoch 3/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.4418 - accuracy: 0.9052
Epoch 4/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.2168 - accuracy: 0.9179
Epoch 5/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.2026 - accuracy: 0.9230
Epoch 6/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1700 - accuracy: 0.9354
Epoch 7/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1865 - accuracy: 0.9452
Epoch 8/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1980 - accuracy: 0.9430
Epoch 9/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1246 - accuracy: 0.9550
Epoch 10/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1021 - accuracy: 0.9613
Epoch 11/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0969 - accuracy: 0.9634
Epoch 12/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0901 - accuracy: 0.9673
Epoch 13/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0806 - accuracy: 0.9695
Epoch 14/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0761 - accuracy: 0.9719
Epoch 15/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0888 - accuracy: 0.9680
Epoch 16/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0788 - accuracy: 0.9725
Epoch 17/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0697 - accuracy: 0.9757
Epoch 18/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0814 - accuracy: 0.9755
Epoch 19/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0628 - accuracy: 0.9775
Epoch 20/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0679 - accuracy: 0.9769
Epoch 21/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0611 - accuracy: 0.9772
Epoch 22/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0597 - accuracy: 0.9781
Epoch 23/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0603 - accuracy: 0.9778
Epoch 24/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0553 - accuracy: 0.9800
Epoch 25/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0608 - accuracy: 0.9788
Epoch 26/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0582 - accuracy: 0.9783
Epoch 27/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0589 - accuracy: 0.9784
Epoch 28/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0531 - accuracy: 0.9801
Epoch 29/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0546 - accuracy: 0.9794
Epoch 30/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0618 - accuracy: 0.9794
Epoch 31/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0526 - accuracy: 0.9801
Epoch 32/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0518 - accuracy: 0.9804
Epoch 33/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0510 - accuracy: 0.9800
Epoch 34/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0500 - accuracy: 0.9807
Epoch 35/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0849 - accuracy: 0.9797
Epoch 36/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0504 - accuracy: 0.9810
Epoch 37/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0496 - accuracy: 0.9813
Epoch 38/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0473 - accuracy: 0.9819
Epoch 39/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0502 - accuracy: 0.9813
Epoch 40/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0636 - accuracy: 0.9776
Epoch 41/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0592 - accuracy: 0.9800
Epoch 42/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0535 - accuracy: 0.9796
Epoch 43/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0695 - accuracy: 0.9809
Epoch 44/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0449 - accuracy: 0.9822
Epoch 45/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0459 - accuracy: 0.9824
Epoch 46/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0459 - accuracy: 0.9822
Epoch 47/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0471 - accuracy: 0.9816
Epoch 48/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0513 - accuracy: 0.9819
Epoch 49/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0442 - accuracy: 0.9830
Epoch 50/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0439 - accuracy: 0.9833
Epoch 51/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0425 - accuracy: 0.9854
Epoch 52/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0436 - accuracy: 0.9836
Epoch 53/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0451 - accuracy: 0.9842
Epoch 54/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0397 - accuracy: 0.9850
Epoch 55/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0395 - accuracy: 0.9859
Epoch 56/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0398 - accuracy: 0.9857
Epoch 57/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0434 - accuracy: 0.9844
Epoch 58/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0481 - accuracy: 0.9831
Epoch 59/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0873 - accuracy: 0.9698
Epoch 60/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0523 - accuracy: 0.9803
Epoch 61/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0447 - accuracy: 0.9825
Epoch 62/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0413 - accuracy: 0.9836
Epoch 63/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0412 - accuracy: 0.9835
Epoch 64/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0437 - accuracy: 0.9836
Epoch 65/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0616 - accuracy: 0.9859
Epoch 66/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0422 - accuracy: 0.9844
Epoch 67/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0378 - accuracy: 0.9862
Epoch 68/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0393 - accuracy: 0.9854
Epoch 69/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0440 - accuracy: 0.9843
Epoch 70/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0356 - accuracy: 0.9864
Epoch 71/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0400 - accuracy: 0.9857
Epoch 72/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0360 - accuracy: 0.9862
Epoch 73/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0326 - accuracy: 0.9874
Epoch 74/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0487 - accuracy: 0.9859
Epoch 75/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0344 - accuracy: 0.9868
Epoch 76/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0388 - accuracy: 0.9867
Epoch 77/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0346 - accuracy: 0.9873
Epoch 78/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0342 - accuracy: 0.9876
Epoch 79/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0325 - accuracy: 0.9871
Epoch 80/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0330 - accuracy: 0.9876
Epoch 81/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0357 - accuracy: 0.9869
Epoch 82/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0335 - accuracy: 0.9871
Epoch 83/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0517 - accuracy: 0.9873
Epoch 84/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0346 - accuracy: 0.9872
Epoch 85/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0369 - accuracy: 0.9857
Epoch 86/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0327 - accuracy: 0.9873
Epoch 87/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0350 - accuracy: 0.9874
Epoch 88/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0360 - accuracy: 0.9866
Epoch 89/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0362 - accuracy: 0.9869
Epoch 90/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0352 - accuracy: 0.9870
Epoch 91/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0329 - accuracy: 0.9876
Epoch 92/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0325 - accuracy: 0.9877
Epoch 93/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0476 - accuracy: 0.9863
Epoch 94/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0316 - accuracy: 0.9877
Epoch 95/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0307 - accuracy: 0.9879
Epoch 96/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0320 - accuracy: 0.9880
Epoch 97/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0324 - accuracy: 0.9875
Epoch 98/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0367 - accuracy: 0.9878
Epoch 99/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0303 - accuracy: 0.9886
Epoch 100/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0480 - accuracy: 0.9874
Epoch 101/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0321 - accuracy: 0.9885
Epoch 102/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0299 - accuracy: 0.9889
Epoch 103/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0312 - accuracy: 0.9880
Epoch 104/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0316 - accuracy: 0.9884
Epoch 105/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0327 - accuracy: 0.9881
Epoch 106/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0317 - accuracy: 0.9881
Epoch 107/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0652 - accuracy: 0.9870
Epoch 108/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0299 - accuracy: 0.9886
Epoch 109/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0322 - accuracy: 0.9880
Epoch 110/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0318 - accuracy: 0.9878
Epoch 111/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0328 - accuracy: 0.9879
Epoch 112/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0295 - accuracy: 0.9890
Epoch 113/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0332 - accuracy: 0.9879
Epoch 114/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0325 - accuracy: 0.9880
Epoch 115/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0310 - accuracy: 0.9883
Epoch 116/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1101 - accuracy: 0.9851
Epoch 117/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0281 - accuracy: 0.9899
Epoch 118/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0309 - accuracy: 0.9883
Epoch 119/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0318 - accuracy: 0.9879
Epoch 120/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0315 - accuracy: 0.9877
Epoch 121/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0337 - accuracy: 0.9885
Epoch 122/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0303 - accuracy: 0.9887
Epoch 123/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0307 - accuracy: 0.9880
Epoch 124/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0308 - accuracy: 0.9885
Epoch 125/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0302 - accuracy: 0.9883
Epoch 126/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0289 - accuracy: 0.9888
Epoch 127/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0294 - accuracy: 0.9884
Epoch 128/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0317 - accuracy: 0.9886
Epoch 129/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0308 - accuracy: 0.9882
Epoch 130/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0315 - accuracy: 0.9888
Epoch 131/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1284 - accuracy: 0.9676
Epoch 132/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0309 - accuracy: 0.9880
Epoch 133/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0324 - accuracy: 0.9876
Epoch 134/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0473 - accuracy: 0.9877
Epoch 135/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0308 - accuracy: 0.9881
Epoch 136/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0317 - accuracy: 0.9874
Epoch 137/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0406 - accuracy: 0.9872
Epoch 138/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0337 - accuracy: 0.9875
Epoch 139/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0281 - accuracy: 0.9889
Epoch 140/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0312 - accuracy: 0.9879
Epoch 141/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0308 - accuracy: 0.9882
Epoch 142/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0292 - accuracy: 0.9887
Epoch 143/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0295 - accuracy: 0.9887
Epoch 144/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0311 - accuracy: 0.9881
Epoch 145/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0290 - accuracy: 0.9885
Epoch 146/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0345 - accuracy: 0.9880
Epoch 147/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0335 - accuracy: 0.9880
Epoch 148/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0324 - accuracy: 0.9879
Epoch 149/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0568 - accuracy: 0.9888
Epoch 150/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0291 - accuracy: 0.9886

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_32_128_64_32_1_150_epochs", model=model, model_history=history)

Observations

The current neural network has four layers with 32, 128, 64, 32 and 1 neurons in each layer. After being trained for 150 epochs, the model achieved an accuracy of 0.9886, it showed signs of overfitting or convergence. As a result, it may be beneficial to stick with 150 epochs and incerase the network complexity by adding more layers/neurons to achieve 100 percent accuracy.



Model Building with multiple layers and neurons (32-128-128-64-32-1) and 150 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(32, activation="relu", input_dim=data_frame.shape[1] - 1))
model.add(Dense(128, activation="relu"))
model.add(Dense(128, activation="relu"))
model.add(Dense(64, activation="relu"))
model.add(Dense(32, activation="relu"))
model.add(Dense(1, activation="sigmoid"))


# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=X_smote, y=y_smote, epochs=150, verbose=1)
Epoch 1/150
2798/2798 [==============================] - 5s 1ms/step - loss: 5.1992 - accuracy: 0.8255
Epoch 2/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.4606 - accuracy: 0.8828
Epoch 3/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.2612 - accuracy: 0.8974
Epoch 4/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.2377 - accuracy: 0.9104
Epoch 5/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1862 - accuracy: 0.9295
Epoch 6/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.2359 - accuracy: 0.9304
Epoch 7/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1471 - accuracy: 0.9428
Epoch 8/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1299 - accuracy: 0.9494
Epoch 9/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1092 - accuracy: 0.9567
Epoch 10/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1016 - accuracy: 0.9601
Epoch 11/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0915 - accuracy: 0.9645
Epoch 12/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0821 - accuracy: 0.9694
Epoch 13/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0783 - accuracy: 0.9718
Epoch 14/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0673 - accuracy: 0.9751
Epoch 15/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0676 - accuracy: 0.9772
Epoch 16/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0966 - accuracy: 0.9773
Epoch 17/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0555 - accuracy: 0.9802
Epoch 18/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0532 - accuracy: 0.9801
Epoch 19/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0654 - accuracy: 0.9760
Epoch 20/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0515 - accuracy: 0.9815
Epoch 21/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0475 - accuracy: 0.9823
Epoch 22/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0675 - accuracy: 0.9796
Epoch 23/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0448 - accuracy: 0.9832
Epoch 24/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0480 - accuracy: 0.9825
Epoch 25/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0412 - accuracy: 0.9847
Epoch 26/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0400 - accuracy: 0.9846
Epoch 27/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0401 - accuracy: 0.9852
Epoch 28/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0409 - accuracy: 0.9857
Epoch 29/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0346 - accuracy: 0.9863
Epoch 30/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0379 - accuracy: 0.9857
Epoch 31/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0383 - accuracy: 0.9856
Epoch 32/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0324 - accuracy: 0.9868
Epoch 33/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0367 - accuracy: 0.9858
Epoch 34/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0341 - accuracy: 0.9863
Epoch 35/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0363 - accuracy: 0.9863
Epoch 36/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0534 - accuracy: 0.9844
Epoch 37/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0326 - accuracy: 0.9864
Epoch 38/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0352 - accuracy: 0.9864
Epoch 39/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0325 - accuracy: 0.9875
Epoch 40/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0370 - accuracy: 0.9862
Epoch 41/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0320 - accuracy: 0.9872
Epoch 42/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0346 - accuracy: 0.9867
Epoch 43/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0311 - accuracy: 0.9879
Epoch 44/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0387 - accuracy: 0.9874
Epoch 45/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0358 - accuracy: 0.9862
Epoch 46/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0314 - accuracy: 0.9879
Epoch 47/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0321 - accuracy: 0.9870
Epoch 48/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0317 - accuracy: 0.9873
Epoch 49/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0323 - accuracy: 0.9876
Epoch 50/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0454 - accuracy: 0.9857
Epoch 51/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0320 - accuracy: 0.9873
Epoch 52/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0313 - accuracy: 0.9879
Epoch 53/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0314 - accuracy: 0.9877
Epoch 54/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0312 - accuracy: 0.9877
Epoch 55/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1440 - accuracy: 0.9648
Epoch 56/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1665 - accuracy: 0.9339
Epoch 57/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1385 - accuracy: 0.9355
Epoch 58/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1208 - accuracy: 0.9534
Epoch 59/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1020 - accuracy: 0.9593
Epoch 60/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0873 - accuracy: 0.9654
Epoch 61/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0736 - accuracy: 0.9736
Epoch 62/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1339 - accuracy: 0.9265
Epoch 63/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1818 - accuracy: 0.8930
Epoch 64/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0686 - accuracy: 0.9728
Epoch 65/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0749 - accuracy: 0.9773
Epoch 66/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0474 - accuracy: 0.9821
Epoch 67/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0470 - accuracy: 0.9813
Epoch 68/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0449 - accuracy: 0.9825
Epoch 69/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1016 - accuracy: 0.9572
Epoch 70/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0531 - accuracy: 0.9800
Epoch 71/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0864 - accuracy: 0.9754
Epoch 72/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0826 - accuracy: 0.9656
Epoch 73/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0915 - accuracy: 0.9539
Epoch 74/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1247 - accuracy: 0.9461
Epoch 75/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0941 - accuracy: 0.9595
Epoch 76/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0517 - accuracy: 0.9822
Epoch 77/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0410 - accuracy: 0.9840
Epoch 78/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0767 - accuracy: 0.9801
Epoch 79/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0415 - accuracy: 0.9855
Epoch 80/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0408 - accuracy: 0.9843
Epoch 81/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0391 - accuracy: 0.9852
Epoch 82/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0407 - accuracy: 0.9843
Epoch 83/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0370 - accuracy: 0.9856
Epoch 84/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0508 - accuracy: 0.9809
Epoch 85/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0432 - accuracy: 0.9845
Epoch 86/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0446 - accuracy: 0.9834
Epoch 87/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0546 - accuracy: 0.9846
Epoch 88/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0371 - accuracy: 0.9856
Epoch 89/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0436 - accuracy: 0.9835
Epoch 90/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0359 - accuracy: 0.9852
Epoch 91/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0539 - accuracy: 0.9852
Epoch 92/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0352 - accuracy: 0.9863
Epoch 93/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0367 - accuracy: 0.9851
Epoch 94/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0369 - accuracy: 0.9861
Epoch 95/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0427 - accuracy: 0.9846
Epoch 96/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0328 - accuracy: 0.9867
Epoch 97/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0331 - accuracy: 0.9865
Epoch 98/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.5963 - accuracy: 0.6070
Epoch 99/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.6425 - accuracy: 0.5683
Epoch 100/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.6377 - accuracy: 0.5737
Epoch 101/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.6081 - accuracy: 0.6021
Epoch 102/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1579 - accuracy: 0.9365
Epoch 103/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1309 - accuracy: 0.9532
Epoch 104/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1037 - accuracy: 0.9577
Epoch 105/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1005 - accuracy: 0.9592
Epoch 106/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0953 - accuracy: 0.9614
Epoch 107/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0868 - accuracy: 0.9657
Epoch 108/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0784 - accuracy: 0.9691
Epoch 109/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0813 - accuracy: 0.9689
Epoch 110/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0621 - accuracy: 0.9756
Epoch 111/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0553 - accuracy: 0.9795
Epoch 112/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1684 - accuracy: 0.9783
Epoch 113/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0542 - accuracy: 0.9797
Epoch 114/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0543 - accuracy: 0.9798
Epoch 115/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0508 - accuracy: 0.9823
Epoch 116/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0488 - accuracy: 0.9823
Epoch 117/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0456 - accuracy: 0.9833
Epoch 118/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0441 - accuracy: 0.9836
Epoch 119/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0510 - accuracy: 0.9827
Epoch 120/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0503 - accuracy: 0.9817
Epoch 121/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0432 - accuracy: 0.9843
Epoch 122/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0410 - accuracy: 0.9844
Epoch 123/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0433 - accuracy: 0.9835
Epoch 124/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0421 - accuracy: 0.9846
Epoch 125/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0411 - accuracy: 0.9844
Epoch 126/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1111 - accuracy: 0.9833
Epoch 127/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0372 - accuracy: 0.9856
Epoch 128/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0372 - accuracy: 0.9858
Epoch 129/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0376 - accuracy: 0.9856
Epoch 130/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0399 - accuracy: 0.9851
Epoch 131/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0414 - accuracy: 0.9851
Epoch 132/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1020 - accuracy: 0.9864
Epoch 133/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0366 - accuracy: 0.9856
Epoch 134/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0371 - accuracy: 0.9860
Epoch 135/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0632 - accuracy: 0.9839
Epoch 136/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0369 - accuracy: 0.9863
Epoch 137/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0371 - accuracy: 0.9860
Epoch 138/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0376 - accuracy: 0.9865
Epoch 139/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0350 - accuracy: 0.9865
Epoch 140/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0356 - accuracy: 0.9862
Epoch 141/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.1184 - accuracy: 0.9856
Epoch 142/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0373 - accuracy: 0.9860
Epoch 143/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0348 - accuracy: 0.9866
Epoch 144/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0346 - accuracy: 0.9867
Epoch 145/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0345 - accuracy: 0.9866
Epoch 146/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0353 - accuracy: 0.9862
Epoch 147/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0533 - accuracy: 0.9862
Epoch 148/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0525 - accuracy: 0.9869
Epoch 149/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0340 - accuracy: 0.9871
Epoch 150/150
2798/2798 [==============================] - 4s 1ms/step - loss: 0.0355 - accuracy: 0.9870

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_32_128_128_64_32_1_150_epochs", model=model, model_history=history)

Observations

The current neural network has four layers with 32, 128, 128, 64, 32 and 1 neurons in each layer. After being trained for 150 epochs, the model achieved an accuracy of 0.9870, it showed signs of overfitting or convergence. As a result, it may be beneficial to stick with 150 epochs and incerase the network complexity by adding more layers/neurons to achieve 100 percent accuracy.



Overfitting - 100 percent accuracy

Model Building with multiple layers and neurons (128-128-128-128-128-1) and 50 epochs

# Let's create a keras sequential model
model = Sequential()

# Let's add dense layers to the sequential model network

model.add(Dense(128, activation="relu", input_dim=data_frame.shape[1]))
model.add(Dense(128, activation="relu"))
model.add(Dense(128, activation="relu"))
model.add(Dense(128, activation="relu"))
model.add(Dense(128, activation="relu"))
model.add(Dense(1, activation="sigmoid"))

# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
# Let's fit the sequential model with input features and output label
history = model.fit(x=np.hstack([X_smote, y_smote.reshape(-1, 1)]), y=y_smote, epochs=50, verbose=1)
Epoch 1/50
2798/2798 [==============================] - 5s 2ms/step - loss: 1.9769 - accuracy: 0.8468
Epoch 2/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.2933 - accuracy: 0.8964
Epoch 3/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.2137 - accuracy: 0.9192
Epoch 4/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.2357 - accuracy: 0.9366
Epoch 5/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.1219 - accuracy: 0.9556
Epoch 6/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0798 - accuracy: 0.9722
Epoch 7/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0784 - accuracy: 0.9764
Epoch 8/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0562 - accuracy: 0.9814
Epoch 9/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0605 - accuracy: 0.9810
Epoch 10/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0440 - accuracy: 0.9856
Epoch 11/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0458 - accuracy: 0.9856
Epoch 12/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0450 - accuracy: 0.9865
Epoch 13/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0686 - accuracy: 0.9883
Epoch 14/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0386 - accuracy: 0.9886
Epoch 15/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0564 - accuracy: 0.9845
Epoch 16/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0419 - accuracy: 0.9884
Epoch 17/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0554 - accuracy: 0.9899
Epoch 18/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0325 - accuracy: 0.9900
Epoch 19/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0313 - accuracy: 0.9908
Epoch 20/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0325 - accuracy: 0.9897
Epoch 21/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0935 - accuracy: 0.9787
Epoch 22/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0434 - accuracy: 0.9881
Epoch 23/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0347 - accuracy: 0.9900
Epoch 24/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0365 - accuracy: 0.9896
Epoch 25/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0302 - accuracy: 0.9914
Epoch 26/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0756 - accuracy: 0.9846
Epoch 27/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.1057 - accuracy: 0.9812
Epoch 28/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0636 - accuracy: 0.9811
Epoch 29/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0559 - accuracy: 0.9841
Epoch 30/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0364 - accuracy: 0.9907
Epoch 31/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0319 - accuracy: 0.9919
Epoch 32/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0521 - accuracy: 0.9932
Epoch 33/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0279 - accuracy: 0.9922
Epoch 34/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0372 - accuracy: 0.9912
Epoch 35/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0380 - accuracy: 0.9910
Epoch 36/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0245 - accuracy: 0.9932
Epoch 37/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0271 - accuracy: 0.9934
Epoch 38/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0189 - accuracy: 0.9944
Epoch 39/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0289 - accuracy: 0.9932
Epoch 40/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0198 - accuracy: 0.9948
Epoch 41/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0278 - accuracy: 0.9938
Epoch 42/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0217 - accuracy: 0.9946
Epoch 43/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0486 - accuracy: 0.9878
Epoch 44/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0508 - accuracy: 0.9867
Epoch 45/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0559 - accuracy: 0.9876
Epoch 46/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0465 - accuracy: 0.9884
Epoch 47/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0367 - accuracy: 0.9895
Epoch 48/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0269 - accuracy: 0.9923
Epoch 49/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0333 - accuracy: 0.9917
Epoch 50/50
2798/2798 [==============================] - 4s 2ms/step - loss: 0.0208 - accuracy: 0.9942

# Let's plot the loss and accuracy using model history on the entire dataset
plot_loss_accuracy(model_name="model_32_128_128_64_32_1_150_epochs", model=model, model_history=history)

Observations

The current neural network has four layers with 32, 128, 64, 32 and 1 neurons in each layer. After being trained for 150 epochs, the model achieved an accuracy of 0.9942, it showed signs of overfitting or convergence. As a result, it may be beneficial to stick with 150 epochs.



Phase - 3 Model selection & evaluation



Data Shuffling

# Let's create a dataframe of oversampling values
oversampled_df = pd.concat([pd.DataFrame(X_smote), pd.DataFrame(y_smote)], axis=1)

# Let's shuffle the oversampled dataframe
oversampled_df = oversampled_df.sample(frac=1).reset_index(drop=True)

X_smote = oversampled_df.iloc[:, :-1].values
y_smote = oversampled_df.iloc[:, -1].values

Dividing data into train and validation subsets

def train_test_split(train_ratio):
    # Let's find the total number of unique output labels in the dataset
    unique_labels = list(set(y_smote))
    num_labels = len(unique_labels)

    # Let's create a dictionary to store the indices of each label in the dataset
    label_indices = {}
    for i in range(len(y_smote)):
        label = y_smote[i]
        if label not in label_indices:
            label_indices[label] = []
        label_indices[label].append(i)

    # Let's shuffle the indices of each label in the dictionary
    for label in unique_labels:
        random.shuffle(label_indices[label])

    # Let's calculate the number of samples needed for each label 
    # to have an equal distribution in both train and test sets

    samples_per_label = len(X_smote) // len(unique_labels)
    train_samples_per_label = int(samples_per_label * train_ratio)
    validation_samples_per_label = samples_per_label - train_samples_per_label

    # Let's initialize empty lists to store the indices of samples in the train and test sets
    train_indices = []
    test_indices = []

    # Let's iterate through the shuffled indices of each label and 
    # allocate the appropriate number of samples for each label to the train 
    # and test sets while maintaining stratification
    for label in unique_labels:
        label_indices_list = label_indices[label]
        for i in range(len(label_indices_list)):
            if i < train_samples_per_label:
                train_indices.append(label_indices_list[i])
            else:
                test_indices.append(label_indices_list[i])
        if len(train_indices) == train_samples_per_label * len(unique_labels):
            break

    # Let's concatenate the input and output data for each set 
    # using the allocated indices to create two datasets with stratification
    train_inputs = np.array([X_smote[i] for i in train_indices])
    train_outputs = np.array([y_smote[i] for i in train_indices])

    test_inputs = np.array([X_smote[i] for i in test_indices])
    test_outputs = np.array([y_smote[i] for i in test_indices])

    return train_inputs, train_outputs, test_inputs, test_outputs
# Let's initialize the train_ratio value
train_ratio = 0.8

# Let's call the train_test_split function to split the data into train and validation datasets
train_inputs, train_outputs, test_inputs, test_outputs = train_test_split(train_ratio=train_ratio)



Model Building

  1. one layer and one neuron (1-1)
  2. Two layers with 8 and 1 neurons respectively(8-1)
  3. Three layers with 4, 4 and 1 neurons respectively(4-4-1)
  4. Three layers with 16, 8 and 1 neurons respectively(16-8-1)
  5. Three layers with 32, 16 and 1 neurons respectively(32-16-1)
  6. Four layers with 128, 128, 64 and 1 neurons respectively(128-128-64-1)
  7. Five layers with 16, 32, 32, 16 and 1 neurons respectively(16-32-32-16-1)
  8. Five layers with 32, 128, 64, 32 and 1 neurons respectively(32-128-64-32-1)
# Let's craete a model with one layer and one neuron
model_base = Sequential()
model_base.add(Dense(1, input_dim=data_frame.shape[1] - 1, activation="sigmoid"))
# Let's create a model with two layers,consisting
# of 8 and 1 neurons, respectively

model_8_1 = Sequential()
model_8_1.add(Dense(8, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_8_1.add(Dense(1, activation="sigmoid"))
# Let's create a model with three layers,consisting
# of 4, 4 and 1 neurons, respectively

model_4_4_1 = Sequential()
model_4_4_1.add(Dense(4, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_4_4_1.add(Dense(4, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_4_4_1.add(Dense(1, activation="sigmoid"))
# Let's create a model with three layers,consisting
# of 16, 8 and 1 neurons, respectively

model_16_8_1 = Sequential()
model_16_8_1.add(Dense(16, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_16_8_1.add(Dense(8, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_16_8_1.add(Dense(1, activation="sigmoid"))
# Let's create a model with three layers,consisting
# of 32, 16 and 1 neurons, respectively

model_32_16_1 = Sequential()
model_32_16_1.add(Dense(32, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_32_16_1.add(Dense(16, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_32_16_1.add(Dense(1, activation="sigmoid"))
# Let's create a model with three layers,consisting
# of 128, 128, 64 and 1 neurons, respectively

model_128_128_64_1 = Sequential()
model_128_128_64_1.add(Dense(128, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_128_128_64_1.add(Dense(128, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_128_128_64_1.add(Dense(64, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_128_128_64_1.add(Dense(1, activation="sigmoid"))
# Let's create a model with three layers,consisting
# of 16, 32, 32, 16 and 1 neurons, respectively

model_16_32_32_16_1 = Sequential()
model_16_32_32_16_1.add(Dense(16, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_16_32_32_16_1.add(Dense(32, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_16_32_32_16_1.add(Dense(32, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_16_32_32_16_1.add(Dense(16, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_16_32_32_16_1.add(Dense(1, activation="sigmoid"))
# Let's create a model with three layers,consisting
# of 32, 128, 64, 32 and 1 neurons, respectively

model_32_128_64_32_1 = Sequential()
model_32_128_64_32_1.add(Dense(32, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_32_128_64_32_1.add(Dense(128, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_32_128_64_32_1.add(Dense(64, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_32_128_64_32_1.add(Dense(32, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_32_128_64_32_1.add(Dense(1, activation="sigmoid"))
# Let's initialize a list which has all the model names corresponding to adam optimizer
adam_model_names = [
    "model_base",
    "model_8_1", 
    "model_4_4_1", 
    "model_16_8_1", 
    "model_32_16_1", 
    "model_128_128_64_1",
    "model_16_32_32_16_1",
    "model_32_128_64_32_1"
]

# Let's initialize a list to store all the compiled models during training and evaluation
adam_models = []

# Let's build the model using binary_crossentropy as the loss function 
# and accuracy as the evaluation metric during the compilation process
model_base.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
model_8_1.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
model_4_4_1.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
model_16_8_1.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
model_32_16_1.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
model_128_128_64_1.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
model_16_32_32_16_1.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])
model_32_128_64_32_1.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])

# Let's append the model_objects into a list
for model_name in adam_model_names:
    adam_models.append(globals()[model_name])
# Let's initialize a list to store all the model histories during training and evaluation
adam_history = []

for index, model_name in enumerate(adam_model_names):
    print()
    print(f"Let's fit the sequential model --> {model_name}")
    # print()
    callback_file = f"models/adam_{model_name}.hdf5"
    callback_a = ModelCheckpoint(filepath = callback_file, monitor="val_loss", save_best_only = True, save_weights_only = True, verbose = 0)
    callback_b = EarlyStopping(monitor="val_loss", mode="min", patience=20, verbose=0)
    history = adam_models[index].fit(x=train_inputs, y=train_outputs, validation_data=(test_inputs, test_outputs), epochs = 100, batch_size=128, callbacks = [callback_a, callback_b], verbose = 1)
    adam_history.append(history)

Let's fit the sequential model --> model_base
Epoch 1/100
560/560 [==============================] - 1s 1ms/step - loss: 115.1353 - accuracy: 0.2126 - val_loss: 34.9231 - val_accuracy: 0.1891
Epoch 2/100
560/560 [==============================] - 1s 1ms/step - loss: 18.1631 - accuracy: 0.2462 - val_loss: 9.2389 - val_accuracy: 0.2576
Epoch 3/100
560/560 [==============================] - 1s 1ms/step - loss: 4.8010 - accuracy: 0.3860 - val_loss: 1.6328 - val_accuracy: 0.6013
Epoch 4/100
560/560 [==============================] - 1s 1ms/step - loss: 1.1116 - accuracy: 0.6815 - val_loss: 1.0140 - val_accuracy: 0.7274
Epoch 5/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8715 - accuracy: 0.7695 - val_loss: 0.6828 - val_accuracy: 0.8080
Epoch 6/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7638 - accuracy: 0.7873 - val_loss: 1.2743 - val_accuracy: 0.7924
Epoch 7/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9047 - accuracy: 0.7792 - val_loss: 0.4623 - val_accuracy: 0.8138
Epoch 8/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6571 - accuracy: 0.7986 - val_loss: 0.4339 - val_accuracy: 0.8927
Epoch 9/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6637 - accuracy: 0.8065 - val_loss: 0.3809 - val_accuracy: 0.8075
Epoch 10/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7673 - accuracy: 0.7982 - val_loss: 0.3293 - val_accuracy: 0.8781
Epoch 11/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6256 - accuracy: 0.8122 - val_loss: 0.4525 - val_accuracy: 0.8104
Epoch 12/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5292 - accuracy: 0.8255 - val_loss: 0.4427 - val_accuracy: 0.8124
Epoch 13/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6576 - accuracy: 0.8093 - val_loss: 0.4474 - val_accuracy: 0.9052
Epoch 14/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6552 - accuracy: 0.8086 - val_loss: 0.8844 - val_accuracy: 0.5664
Epoch 15/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6786 - accuracy: 0.8044 - val_loss: 1.6348 - val_accuracy: 0.8066
Epoch 16/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6974 - accuracy: 0.8080 - val_loss: 0.9252 - val_accuracy: 0.8123
Epoch 17/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5813 - accuracy: 0.8269 - val_loss: 0.5368 - val_accuracy: 0.8094
Epoch 18/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6800 - accuracy: 0.8158 - val_loss: 0.9807 - val_accuracy: 0.7973
Epoch 19/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6311 - accuracy: 0.8149 - val_loss: 0.4327 - val_accuracy: 0.8233
Epoch 20/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6322 - accuracy: 0.8152 - val_loss: 0.4548 - val_accuracy: 0.8328
Epoch 21/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6186 - accuracy: 0.8186 - val_loss: 0.4580 - val_accuracy: 0.8218
Epoch 22/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6435 - accuracy: 0.8127 - val_loss: 0.5552 - val_accuracy: 0.8192
Epoch 23/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6136 - accuracy: 0.8240 - val_loss: 0.7626 - val_accuracy: 0.8050
Epoch 24/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6508 - accuracy: 0.8129 - val_loss: 0.3898 - val_accuracy: 0.8303
Epoch 25/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6374 - accuracy: 0.8164 - val_loss: 1.1220 - val_accuracy: 0.6105
Epoch 26/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6581 - accuracy: 0.8197 - val_loss: 0.4709 - val_accuracy: 0.8700
Epoch 27/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6010 - accuracy: 0.8225 - val_loss: 0.4799 - val_accuracy: 0.8213
Epoch 28/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6418 - accuracy: 0.8185 - val_loss: 0.8431 - val_accuracy: 0.8164
Epoch 29/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6533 - accuracy: 0.8118 - val_loss: 0.6110 - val_accuracy: 0.8089
Epoch 30/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7114 - accuracy: 0.8130 - val_loss: 0.9167 - val_accuracy: 0.8214

Let's fit the sequential model --> model_8_1
Epoch 1/100
560/560 [==============================] - 1s 1ms/step - loss: 240.0401 - accuracy: 0.8344 - val_loss: 1.8173 - val_accuracy: 0.8932
Epoch 2/100
560/560 [==============================] - 1s 1ms/step - loss: 1.6871 - accuracy: 0.8867 - val_loss: 1.4522 - val_accuracy: 0.9051
Epoch 3/100
560/560 [==============================] - 1s 1ms/step - loss: 1.5890 - accuracy: 0.8875 - val_loss: 3.4997 - val_accuracy: 0.8431
Epoch 4/100
560/560 [==============================] - 1s 1ms/step - loss: 1.5462 - accuracy: 0.8870 - val_loss: 1.0947 - val_accuracy: 0.9090
Epoch 5/100
560/560 [==============================] - 1s 1ms/step - loss: 1.4668 - accuracy: 0.8899 - val_loss: 1.0669 - val_accuracy: 0.9025
Epoch 6/100
560/560 [==============================] - 1s 1ms/step - loss: 1.4694 - accuracy: 0.8900 - val_loss: 2.8459 - val_accuracy: 0.8542
Epoch 7/100
560/560 [==============================] - 1s 1ms/step - loss: 1.4570 - accuracy: 0.8886 - val_loss: 0.9409 - val_accuracy: 0.9100
Epoch 8/100
560/560 [==============================] - 1s 1ms/step - loss: 1.3384 - accuracy: 0.8930 - val_loss: 1.4429 - val_accuracy: 0.8551
Epoch 9/100
560/560 [==============================] - 1s 1ms/step - loss: 1.2625 - accuracy: 0.8941 - val_loss: 0.7841 - val_accuracy: 0.9039
Epoch 10/100
560/560 [==============================] - 1s 1ms/step - loss: 1.1851 - accuracy: 0.8939 - val_loss: 1.8956 - val_accuracy: 0.8709
Epoch 11/100
560/560 [==============================] - 1s 1ms/step - loss: 1.3390 - accuracy: 0.8940 - val_loss: 1.2162 - val_accuracy: 0.8583
Epoch 12/100
560/560 [==============================] - 1s 1ms/step - loss: 1.1275 - accuracy: 0.8974 - val_loss: 0.7647 - val_accuracy: 0.8866
Epoch 13/100
560/560 [==============================] - 1s 1ms/step - loss: 1.0370 - accuracy: 0.8997 - val_loss: 0.8939 - val_accuracy: 0.9263
Epoch 14/100
560/560 [==============================] - 1s 1ms/step - loss: 1.0523 - accuracy: 0.8966 - val_loss: 1.0806 - val_accuracy: 0.8633
Epoch 15/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9827 - accuracy: 0.8983 - val_loss: 1.2989 - val_accuracy: 0.8534
Epoch 16/100
560/560 [==============================] - 1s 1ms/step - loss: 1.0315 - accuracy: 0.8958 - val_loss: 0.6221 - val_accuracy: 0.9199
Epoch 17/100
560/560 [==============================] - 1s 1ms/step - loss: 1.0736 - accuracy: 0.8940 - val_loss: 1.0760 - val_accuracy: 0.8577
Epoch 18/100
560/560 [==============================] - 1s 1ms/step - loss: 1.0195 - accuracy: 0.8971 - val_loss: 0.5595 - val_accuracy: 0.9222
Epoch 19/100
560/560 [==============================] - 1s 1ms/step - loss: 1.1137 - accuracy: 0.8930 - val_loss: 0.6551 - val_accuracy: 0.9312
Epoch 20/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9347 - accuracy: 0.8978 - val_loss: 0.7491 - val_accuracy: 0.8752
Epoch 21/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9380 - accuracy: 0.8957 - val_loss: 1.8218 - val_accuracy: 0.8437
Epoch 22/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9003 - accuracy: 0.8977 - val_loss: 0.5886 - val_accuracy: 0.9313
Epoch 23/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9088 - accuracy: 0.8976 - val_loss: 0.7597 - val_accuracy: 0.8694
Epoch 24/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8968 - accuracy: 0.8979 - val_loss: 0.8414 - val_accuracy: 0.8640
Epoch 25/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9955 - accuracy: 0.8937 - val_loss: 0.4804 - val_accuracy: 0.9008
Epoch 26/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8805 - accuracy: 0.8992 - val_loss: 0.6117 - val_accuracy: 0.8778
Epoch 27/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8877 - accuracy: 0.8954 - val_loss: 0.4681 - val_accuracy: 0.8991
Epoch 28/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8373 - accuracy: 0.8974 - val_loss: 0.6102 - val_accuracy: 0.8751
Epoch 29/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8085 - accuracy: 0.8988 - val_loss: 1.4226 - val_accuracy: 0.8449
Epoch 30/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8011 - accuracy: 0.9022 - val_loss: 0.5150 - val_accuracy: 0.8917
Epoch 31/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8890 - accuracy: 0.8961 - val_loss: 2.1533 - val_accuracy: 0.8547
Epoch 32/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8531 - accuracy: 0.8970 - val_loss: 0.4246 - val_accuracy: 0.9115
Epoch 33/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8895 - accuracy: 0.8949 - val_loss: 0.4203 - val_accuracy: 0.9182
Epoch 34/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7541 - accuracy: 0.9013 - val_loss: 1.1752 - val_accuracy: 0.8801
Epoch 35/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8869 - accuracy: 0.8964 - val_loss: 0.4961 - val_accuracy: 0.8955
Epoch 36/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7478 - accuracy: 0.9003 - val_loss: 0.8869 - val_accuracy: 0.9076
Epoch 37/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8794 - accuracy: 0.8964 - val_loss: 0.6163 - val_accuracy: 0.9362
Epoch 38/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7616 - accuracy: 0.9022 - val_loss: 1.0523 - val_accuracy: 0.8880
Epoch 39/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8667 - accuracy: 0.8961 - val_loss: 0.4023 - val_accuracy: 0.9254
Epoch 40/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8597 - accuracy: 0.8967 - val_loss: 1.1130 - val_accuracy: 0.8859
Epoch 41/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8283 - accuracy: 0.9001 - val_loss: 0.6693 - val_accuracy: 0.9269
Epoch 42/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6937 - accuracy: 0.9029 - val_loss: 0.6140 - val_accuracy: 0.8752
Epoch 43/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8782 - accuracy: 0.8948 - val_loss: 0.5738 - val_accuracy: 0.9345
Epoch 44/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7561 - accuracy: 0.9012 - val_loss: 0.6848 - val_accuracy: 0.8714
Epoch 45/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7685 - accuracy: 0.9000 - val_loss: 0.8468 - val_accuracy: 0.9097
Epoch 46/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8573 - accuracy: 0.8957 - val_loss: 0.3487 - val_accuracy: 0.9168
Epoch 47/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7604 - accuracy: 0.9018 - val_loss: 0.3934 - val_accuracy: 0.8992
Epoch 48/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7294 - accuracy: 0.9005 - val_loss: 0.4474 - val_accuracy: 0.8901
Epoch 49/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6510 - accuracy: 0.9063 - val_loss: 2.6165 - val_accuracy: 0.8161
Epoch 50/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8297 - accuracy: 0.8980 - val_loss: 0.7146 - val_accuracy: 0.8631
Epoch 51/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7451 - accuracy: 0.9005 - val_loss: 0.5393 - val_accuracy: 0.9358
Epoch 52/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6517 - accuracy: 0.9061 - val_loss: 0.3296 - val_accuracy: 0.9324
Epoch 53/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7529 - accuracy: 0.8992 - val_loss: 0.3333 - val_accuracy: 0.9152
Epoch 54/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7536 - accuracy: 0.9003 - val_loss: 2.2379 - val_accuracy: 0.8199
Epoch 55/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8108 - accuracy: 0.8984 - val_loss: 0.4996 - val_accuracy: 0.9363
Epoch 56/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6903 - accuracy: 0.9053 - val_loss: 0.7522 - val_accuracy: 0.8572
Epoch 57/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6888 - accuracy: 0.9020 - val_loss: 0.2958 - val_accuracy: 0.9269
Epoch 58/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7343 - accuracy: 0.9004 - val_loss: 1.3552 - val_accuracy: 0.8694
Epoch 59/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6555 - accuracy: 0.9053 - val_loss: 0.4452 - val_accuracy: 0.9387
Epoch 60/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7874 - accuracy: 0.9006 - val_loss: 0.4130 - val_accuracy: 0.8938
Epoch 61/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7151 - accuracy: 0.9029 - val_loss: 0.4516 - val_accuracy: 0.8872
Epoch 62/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6763 - accuracy: 0.9015 - val_loss: 1.4290 - val_accuracy: 0.8568
Epoch 63/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6980 - accuracy: 0.9040 - val_loss: 0.3724 - val_accuracy: 0.9392
Epoch 64/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5583 - accuracy: 0.9105 - val_loss: 0.6652 - val_accuracy: 0.9143
Epoch 65/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7043 - accuracy: 0.9017 - val_loss: 0.3094 - val_accuracy: 0.9156
Epoch 66/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7149 - accuracy: 0.8993 - val_loss: 0.5642 - val_accuracy: 0.8703
Epoch 67/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6561 - accuracy: 0.9052 - val_loss: 0.3082 - val_accuracy: 0.9354
Epoch 68/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7683 - accuracy: 0.8993 - val_loss: 0.3632 - val_accuracy: 0.9419
Epoch 69/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7310 - accuracy: 0.9000 - val_loss: 0.8201 - val_accuracy: 0.8524
Epoch 70/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7012 - accuracy: 0.9007 - val_loss: 0.4052 - val_accuracy: 0.9309
Epoch 71/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6760 - accuracy: 0.9035 - val_loss: 0.5787 - val_accuracy: 0.9244
Epoch 72/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6027 - accuracy: 0.9062 - val_loss: 0.7996 - val_accuracy: 0.9285
Epoch 73/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6892 - accuracy: 0.9008 - val_loss: 1.0883 - val_accuracy: 0.8834
Epoch 74/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7353 - accuracy: 0.8998 - val_loss: 0.7314 - val_accuracy: 0.9108
Epoch 75/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6412 - accuracy: 0.9042 - val_loss: 0.4663 - val_accuracy: 0.9335
Epoch 76/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7359 - accuracy: 0.8957 - val_loss: 0.4034 - val_accuracy: 0.9411
Epoch 77/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6140 - accuracy: 0.9055 - val_loss: 0.4191 - val_accuracy: 0.9411

Let's fit the sequential model --> model_4_4_1
Epoch 1/100
560/560 [==============================] - 2s 2ms/step - loss: 1051.4530 - accuracy: 0.5401 - val_loss: 145.1530 - val_accuracy: 0.5527
Epoch 2/100
560/560 [==============================] - 1s 1ms/step - loss: 35.1919 - accuracy: 0.5005 - val_loss: 0.7609 - val_accuracy: 0.5017
Epoch 3/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7358 - accuracy: 0.5020 - val_loss: 0.7197 - val_accuracy: 0.5017
Epoch 4/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7089 - accuracy: 0.5021 - val_loss: 0.7012 - val_accuracy: 0.5017
Epoch 5/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6966 - accuracy: 0.5022 - val_loss: 0.6941 - val_accuracy: 0.5017
Epoch 6/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6927 - accuracy: 0.5022 - val_loss: 0.6923 - val_accuracy: 0.5017
Epoch 7/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6919 - accuracy: 0.5021 - val_loss: 0.6920 - val_accuracy: 0.5017
Epoch 8/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6917 - accuracy: 0.5010 - val_loss: 0.6920 - val_accuracy: 0.5017
Epoch 9/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6917 - accuracy: 0.5021 - val_loss: 0.6919 - val_accuracy: 0.5017
Epoch 10/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6917 - accuracy: 0.5018 - val_loss: 0.6920 - val_accuracy: 0.5017
Epoch 11/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6917 - accuracy: 0.5013 - val_loss: 0.6919 - val_accuracy: 0.5017
Epoch 12/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6917 - accuracy: 0.5011 - val_loss: 0.6920 - val_accuracy: 0.5017
Epoch 13/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6917 - accuracy: 0.5029 - val_loss: 0.6920 - val_accuracy: 0.5000
Epoch 14/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6917 - accuracy: 0.5014 - val_loss: 0.6919 - val_accuracy: 0.5017
Epoch 15/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.4999 - val_loss: 0.6919 - val_accuracy: 0.5000
Epoch 16/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.4996 - val_loss: 0.6919 - val_accuracy: 0.5018
Epoch 17/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.5011 - val_loss: 0.6919 - val_accuracy: 0.5018
Epoch 18/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.4996 - val_loss: 0.6919 - val_accuracy: 0.5018
Epoch 19/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5015 - val_loss: 0.6918 - val_accuracy: 0.5020
Epoch 20/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5005 - val_loss: 0.6918 - val_accuracy: 0.5000
Epoch 21/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5004 - val_loss: 0.6918 - val_accuracy: 0.5020
Epoch 22/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5016 - val_loss: 0.6918 - val_accuracy: 0.5020
Epoch 23/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5000 - val_loss: 0.6918 - val_accuracy: 0.5020
Epoch 24/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5006 - val_loss: 0.6918 - val_accuracy: 0.5000
Epoch 25/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5004 - val_loss: 0.6918 - val_accuracy: 0.5020
Epoch 26/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6914 - accuracy: 0.5025 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 27/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6914 - accuracy: 0.4998 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 28/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5015 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 29/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5003 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 30/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5019 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 31/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6919 - accuracy: 0.5007 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 32/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.5004 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 33/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.4984 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 34/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.5017 - val_loss: 0.6917 - val_accuracy: 0.5000
Epoch 35/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.4985 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 36/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.4997 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 37/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5024 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 38/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.4994 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 39/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.5001 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 40/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.5011 - val_loss: 0.6917 - val_accuracy: 0.5000
Epoch 41/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.5012 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 42/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.5006 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 43/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.5009 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 44/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.5009 - val_loss: 0.6917 - val_accuracy: 0.5000
Epoch 45/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.5003 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 46/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5011 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 47/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.5006 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 48/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.5022 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 49/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5006 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 50/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.4998 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 51/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.5002 - val_loss: 0.6917 - val_accuracy: 0.5021
Epoch 52/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5029 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 53/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6914 - accuracy: 0.5026 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 54/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6914 - accuracy: 0.5015 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 55/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5011 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 56/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5019 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 57/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6914 - accuracy: 0.4991 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 58/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5021 - val_loss: 0.6916 - val_accuracy: 0.5000
Epoch 59/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6914 - accuracy: 0.4976 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 60/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5012 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 61/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.4997 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 62/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5011 - val_loss: 0.6916 - val_accuracy: 0.5000
Epoch 63/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5002 - val_loss: 0.6915 - val_accuracy: 0.5000
Epoch 64/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6918 - accuracy: 0.5023 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 65/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6915 - accuracy: 0.5026 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 66/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6914 - accuracy: 0.5017 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 67/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6914 - accuracy: 0.5026 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 68/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6914 - accuracy: 0.5011 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 69/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5018 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 70/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5016 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 71/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5013 - val_loss: 0.6916 - val_accuracy: 0.5000
Epoch 72/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5000 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 73/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.4994 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 74/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5044 - val_loss: 0.6916 - val_accuracy: 0.5000
Epoch 75/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5024 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 76/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5010 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 77/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5012 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 78/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5023 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 79/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6914 - accuracy: 0.5002 - val_loss: 0.6915 - val_accuracy: 0.5023
Epoch 80/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5012 - val_loss: 0.6915 - val_accuracy: 0.5023
Epoch 81/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6918 - accuracy: 0.5001 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 82/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5026 - val_loss: 0.6915 - val_accuracy: 0.5000
Epoch 83/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.4993 - val_loss: 0.6915 - val_accuracy: 0.5023
Epoch 84/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6914 - accuracy: 0.5013 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 85/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6914 - accuracy: 0.5011 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 86/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5006 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 87/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5011 - val_loss: 0.6916 - val_accuracy: 0.5023
Epoch 88/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5012 - val_loss: 0.6915 - val_accuracy: 0.5023
Epoch 89/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5030 - val_loss: 0.6915 - val_accuracy: 0.5000
Epoch 90/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.4999 - val_loss: 0.6915 - val_accuracy: 0.5024
Epoch 91/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5014 - val_loss: 0.6915 - val_accuracy: 0.5024
Epoch 92/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5020 - val_loss: 0.6915 - val_accuracy: 0.5024
Epoch 93/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5019 - val_loss: 0.6916 - val_accuracy: 0.5000
Epoch 94/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5004 - val_loss: 0.6915 - val_accuracy: 0.5024
Epoch 95/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.4992 - val_loss: 0.6915 - val_accuracy: 0.5024
Epoch 96/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5002 - val_loss: 0.6915 - val_accuracy: 0.5024
Epoch 97/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5016 - val_loss: 0.6915 - val_accuracy: 0.5000
Epoch 98/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.4985 - val_loss: 0.6915 - val_accuracy: 0.5024
Epoch 99/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5003 - val_loss: 0.6915 - val_accuracy: 0.5023
Epoch 100/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6913 - accuracy: 0.5005 - val_loss: 0.6913 - val_accuracy: 0.5026

Let's fit the sequential model --> model_16_8_1
Epoch 1/100
560/560 [==============================] - 2s 2ms/step - loss: 38.7238 - accuracy: 0.7849 - val_loss: 2.5194 - val_accuracy: 0.8419
Epoch 2/100
560/560 [==============================] - 1s 1ms/step - loss: 2.8236 - accuracy: 0.8098 - val_loss: 1.5368 - val_accuracy: 0.8496
Epoch 3/100
560/560 [==============================] - 1s 1ms/step - loss: 2.3925 - accuracy: 0.8199 - val_loss: 1.2579 - val_accuracy: 0.8504
Epoch 4/100
560/560 [==============================] - 1s 1ms/step - loss: 2.1058 - accuracy: 0.8329 - val_loss: 1.0903 - val_accuracy: 0.8538
Epoch 5/100
560/560 [==============================] - 1s 1ms/step - loss: 1.8470 - accuracy: 0.8390 - val_loss: 3.0336 - val_accuracy: 0.8337
Epoch 6/100
560/560 [==============================] - 1s 1ms/step - loss: 1.9893 - accuracy: 0.8422 - val_loss: 0.7543 - val_accuracy: 0.8944
Epoch 7/100
560/560 [==============================] - 1s 1ms/step - loss: 1.7615 - accuracy: 0.8630 - val_loss: 1.3418 - val_accuracy: 0.9067
Epoch 8/100
560/560 [==============================] - 1s 1ms/step - loss: 1.1681 - accuracy: 0.8943 - val_loss: 0.6564 - val_accuracy: 0.9266
Epoch 9/100
560/560 [==============================] - 1s 1ms/step - loss: 1.7630 - accuracy: 0.8728 - val_loss: 6.1886 - val_accuracy: 0.8075
Epoch 10/100
560/560 [==============================] - 1s 1ms/step - loss: 1.3514 - accuracy: 0.8906 - val_loss: 5.0201 - val_accuracy: 0.7966
Epoch 11/100
560/560 [==============================] - 1s 1ms/step - loss: 1.2655 - accuracy: 0.8922 - val_loss: 2.2677 - val_accuracy: 0.8530
Epoch 12/100
560/560 [==============================] - 1s 1ms/step - loss: 1.3525 - accuracy: 0.8908 - val_loss: 0.9719 - val_accuracy: 0.8817
Epoch 13/100
560/560 [==============================] - 1s 1ms/step - loss: 1.3384 - accuracy: 0.8931 - val_loss: 0.7751 - val_accuracy: 0.9336
Epoch 14/100
560/560 [==============================] - 1s 1ms/step - loss: 1.2772 - accuracy: 0.8981 - val_loss: 1.1699 - val_accuracy: 0.8804
Epoch 15/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9858 - accuracy: 0.9059 - val_loss: 1.9753 - val_accuracy: 0.8429
Epoch 16/100
560/560 [==============================] - 1s 1ms/step - loss: 1.5420 - accuracy: 0.8890 - val_loss: 1.2404 - val_accuracy: 0.8988
Epoch 17/100
560/560 [==============================] - 1s 1ms/step - loss: 1.2266 - accuracy: 0.8983 - val_loss: 0.5460 - val_accuracy: 0.9386
Epoch 18/100
560/560 [==============================] - 1s 1ms/step - loss: 1.1224 - accuracy: 0.9028 - val_loss: 0.7554 - val_accuracy: 0.9020
Epoch 19/100
560/560 [==============================] - 1s 1ms/step - loss: 1.5554 - accuracy: 0.8948 - val_loss: 0.5996 - val_accuracy: 0.9430
Epoch 20/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9858 - accuracy: 0.9061 - val_loss: 1.3245 - val_accuracy: 0.8933
Epoch 21/100
560/560 [==============================] - 1s 1ms/step - loss: 1.0204 - accuracy: 0.9061 - val_loss: 0.5876 - val_accuracy: 0.9382
Epoch 22/100
560/560 [==============================] - 1s 1ms/step - loss: 1.0369 - accuracy: 0.9008 - val_loss: 2.3639 - val_accuracy: 0.8310
Epoch 23/100
560/560 [==============================] - 1s 1ms/step - loss: 1.0024 - accuracy: 0.9057 - val_loss: 0.7236 - val_accuracy: 0.9066
Epoch 24/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8640 - accuracy: 0.9091 - val_loss: 0.4751 - val_accuracy: 0.9462
Epoch 25/100
560/560 [==============================] - 1s 1ms/step - loss: 1.0781 - accuracy: 0.9004 - val_loss: 0.4408 - val_accuracy: 0.9406
Epoch 26/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7595 - accuracy: 0.9155 - val_loss: 0.3373 - val_accuracy: 0.9441
Epoch 27/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9412 - accuracy: 0.9032 - val_loss: 0.5056 - val_accuracy: 0.9174
Epoch 28/100
560/560 [==============================] - 1s 1ms/step - loss: 1.0472 - accuracy: 0.9037 - val_loss: 0.6263 - val_accuracy: 0.9436
Epoch 29/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9692 - accuracy: 0.9080 - val_loss: 2.8056 - val_accuracy: 0.6955
Epoch 30/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7581 - accuracy: 0.9183 - val_loss: 0.5928 - val_accuracy: 0.9396
Epoch 31/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9556 - accuracy: 0.9103 - val_loss: 2.2907 - val_accuracy: 0.8304
Epoch 32/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7860 - accuracy: 0.9235 - val_loss: 0.3329 - val_accuracy: 0.9496
Epoch 33/100
560/560 [==============================] - 1s 1ms/step - loss: 1.0301 - accuracy: 0.9138 - val_loss: 4.5587 - val_accuracy: 0.8080
Epoch 34/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9551 - accuracy: 0.9219 - val_loss: 1.2142 - val_accuracy: 0.9239
Epoch 35/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7103 - accuracy: 0.9297 - val_loss: 0.6890 - val_accuracy: 0.9094
Epoch 36/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5861 - accuracy: 0.9362 - val_loss: 0.1867 - val_accuracy: 0.9648
Epoch 37/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5872 - accuracy: 0.9397 - val_loss: 0.2875 - val_accuracy: 0.9568
Epoch 38/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5485 - accuracy: 0.9444 - val_loss: 0.1998 - val_accuracy: 0.9678
Epoch 39/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5164 - accuracy: 0.9457 - val_loss: 0.1645 - val_accuracy: 0.9704
Epoch 40/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5050 - accuracy: 0.9468 - val_loss: 0.3722 - val_accuracy: 0.9430
Epoch 41/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3952 - accuracy: 0.9541 - val_loss: 1.1757 - val_accuracy: 0.8947
Epoch 42/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4810 - accuracy: 0.9498 - val_loss: 0.2611 - val_accuracy: 0.9572
Epoch 43/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6395 - accuracy: 0.9423 - val_loss: 0.1895 - val_accuracy: 0.9723
Epoch 44/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3940 - accuracy: 0.9584 - val_loss: 0.1722 - val_accuracy: 0.9709
Epoch 45/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6230 - accuracy: 0.9428 - val_loss: 0.7880 - val_accuracy: 0.9181
Epoch 46/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4263 - accuracy: 0.9552 - val_loss: 0.2674 - val_accuracy: 0.9558
Epoch 47/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5983 - accuracy: 0.9539 - val_loss: 0.5462 - val_accuracy: 0.9422
Epoch 48/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4448 - accuracy: 0.9552 - val_loss: 0.3708 - val_accuracy: 0.9592
Epoch 49/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5333 - accuracy: 0.9547 - val_loss: 0.1719 - val_accuracy: 0.9744
Epoch 50/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4370 - accuracy: 0.9577 - val_loss: 0.1112 - val_accuracy: 0.9828
Epoch 51/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4126 - accuracy: 0.9619 - val_loss: 0.1367 - val_accuracy: 0.9822
Epoch 52/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6278 - accuracy: 0.9532 - val_loss: 0.2533 - val_accuracy: 0.9700
Epoch 53/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5721 - accuracy: 0.9529 - val_loss: 0.2851 - val_accuracy: 0.9674
Epoch 54/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4070 - accuracy: 0.9623 - val_loss: 0.1221 - val_accuracy: 0.9827
Epoch 55/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3130 - accuracy: 0.9677 - val_loss: 0.1357 - val_accuracy: 0.9813
Epoch 56/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6493 - accuracy: 0.9490 - val_loss: 0.8613 - val_accuracy: 0.9125
Epoch 57/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3784 - accuracy: 0.9643 - val_loss: 0.1454 - val_accuracy: 0.9790
Epoch 58/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4192 - accuracy: 0.9605 - val_loss: 0.3491 - val_accuracy: 0.9644
Epoch 59/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6036 - accuracy: 0.9511 - val_loss: 0.2113 - val_accuracy: 0.9612
Epoch 60/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3967 - accuracy: 0.9633 - val_loss: 0.1800 - val_accuracy: 0.9759
Epoch 61/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3819 - accuracy: 0.9628 - val_loss: 0.5612 - val_accuracy: 0.9486
Epoch 62/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4441 - accuracy: 0.9600 - val_loss: 0.6070 - val_accuracy: 0.9491
Epoch 63/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5017 - accuracy: 0.9567 - val_loss: 0.6799 - val_accuracy: 0.9440
Epoch 64/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3455 - accuracy: 0.9653 - val_loss: 0.1148 - val_accuracy: 0.9837
Epoch 65/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4841 - accuracy: 0.9576 - val_loss: 0.4883 - val_accuracy: 0.9578
Epoch 66/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3172 - accuracy: 0.9675 - val_loss: 0.1159 - val_accuracy: 0.9840
Epoch 67/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4262 - accuracy: 0.9589 - val_loss: 0.1702 - val_accuracy: 0.9764
Epoch 68/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4748 - accuracy: 0.9604 - val_loss: 0.6698 - val_accuracy: 0.9513
Epoch 69/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3712 - accuracy: 0.9651 - val_loss: 0.8544 - val_accuracy: 0.9018
Epoch 70/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3409 - accuracy: 0.9664 - val_loss: 0.3831 - val_accuracy: 0.9316

Let's fit the sequential model --> model_32_16_1
Epoch 1/100
560/560 [==============================] - 2s 2ms/step - loss: 5.2702 - accuracy: 0.8422 - val_loss: 1.3416 - val_accuracy: 0.8711
Epoch 2/100
560/560 [==============================] - 1s 1ms/step - loss: 1.8264 - accuracy: 0.8744 - val_loss: 0.8919 - val_accuracy: 0.9232
Epoch 3/100
560/560 [==============================] - 1s 1ms/step - loss: 1.5741 - accuracy: 0.8936 - val_loss: 1.2019 - val_accuracy: 0.9153
Epoch 4/100
560/560 [==============================] - 1s 1ms/step - loss: 1.5873 - accuracy: 0.8979 - val_loss: 0.6017 - val_accuracy: 0.9201
Epoch 5/100
560/560 [==============================] - 1s 1ms/step - loss: 1.8817 - accuracy: 0.8941 - val_loss: 3.0219 - val_accuracy: 0.8563
Epoch 6/100
560/560 [==============================] - 1s 1ms/step - loss: 1.4022 - accuracy: 0.9051 - val_loss: 2.9702 - val_accuracy: 0.7302
Epoch 7/100
560/560 [==============================] - 1s 1ms/step - loss: 1.4584 - accuracy: 0.9037 - val_loss: 0.6671 - val_accuracy: 0.9348
Epoch 8/100
560/560 [==============================] - 1s 1ms/step - loss: 1.9055 - accuracy: 0.8991 - val_loss: 0.8742 - val_accuracy: 0.9244
Epoch 9/100
560/560 [==============================] - 1s 1ms/step - loss: 1.2747 - accuracy: 0.9100 - val_loss: 0.8948 - val_accuracy: 0.9179
Epoch 10/100
560/560 [==============================] - 1s 1ms/step - loss: 1.8506 - accuracy: 0.9034 - val_loss: 1.3057 - val_accuracy: 0.8770
Epoch 11/100
560/560 [==============================] - 1s 1ms/step - loss: 1.7255 - accuracy: 0.9043 - val_loss: 1.1840 - val_accuracy: 0.9153
Epoch 12/100
560/560 [==============================] - 1s 1ms/step - loss: 1.4171 - accuracy: 0.9091 - val_loss: 0.4982 - val_accuracy: 0.9476
Epoch 13/100
560/560 [==============================] - 1s 1ms/step - loss: 1.5936 - accuracy: 0.9073 - val_loss: 1.8414 - val_accuracy: 0.9021
Epoch 14/100
560/560 [==============================] - 1s 1ms/step - loss: 1.1967 - accuracy: 0.9176 - val_loss: 1.0905 - val_accuracy: 0.9295
Epoch 15/100
560/560 [==============================] - 1s 1ms/step - loss: 1.3794 - accuracy: 0.9121 - val_loss: 0.8887 - val_accuracy: 0.9401
Epoch 16/100
560/560 [==============================] - 1s 1ms/step - loss: 1.1273 - accuracy: 0.9225 - val_loss: 0.5206 - val_accuracy: 0.9499
Epoch 17/100
560/560 [==============================] - 1s 1ms/step - loss: 1.3017 - accuracy: 0.9140 - val_loss: 0.5463 - val_accuracy: 0.9555
Epoch 18/100
560/560 [==============================] - 1s 1ms/step - loss: 1.1341 - accuracy: 0.9200 - val_loss: 0.5340 - val_accuracy: 0.9469
Epoch 19/100
560/560 [==============================] - 1s 1ms/step - loss: 1.2158 - accuracy: 0.9202 - val_loss: 0.6591 - val_accuracy: 0.9116
Epoch 20/100
560/560 [==============================] - 1s 1ms/step - loss: 1.1463 - accuracy: 0.9202 - val_loss: 0.5202 - val_accuracy: 0.9551
Epoch 21/100
560/560 [==============================] - 1s 1ms/step - loss: 1.1172 - accuracy: 0.9215 - val_loss: 0.6080 - val_accuracy: 0.9496
Epoch 22/100
560/560 [==============================] - 1s 1ms/step - loss: 1.1887 - accuracy: 0.9208 - val_loss: 0.5617 - val_accuracy: 0.9536
Epoch 23/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9905 - accuracy: 0.9266 - val_loss: 0.4643 - val_accuracy: 0.9542
Epoch 24/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6968 - accuracy: 0.9371 - val_loss: 0.4356 - val_accuracy: 0.9251
Epoch 25/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9947 - accuracy: 0.9270 - val_loss: 0.6346 - val_accuracy: 0.9481
Epoch 26/100
560/560 [==============================] - 1s 1ms/step - loss: 1.1520 - accuracy: 0.9276 - val_loss: 0.3267 - val_accuracy: 0.9598
Epoch 27/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8361 - accuracy: 0.9376 - val_loss: 0.2727 - val_accuracy: 0.9644
Epoch 28/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8994 - accuracy: 0.9359 - val_loss: 0.2244 - val_accuracy: 0.9724
Epoch 29/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6789 - accuracy: 0.9429 - val_loss: 0.5114 - val_accuracy: 0.9452
Epoch 30/100
560/560 [==============================] - 1s 1ms/step - loss: 1.0334 - accuracy: 0.9293 - val_loss: 0.3371 - val_accuracy: 0.9600
Epoch 31/100
560/560 [==============================] - 1s 1ms/step - loss: 0.9183 - accuracy: 0.9276 - val_loss: 0.5267 - val_accuracy: 0.9443
Epoch 32/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6948 - accuracy: 0.9384 - val_loss: 0.6009 - val_accuracy: 0.9363
Epoch 33/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7221 - accuracy: 0.9376 - val_loss: 0.2631 - val_accuracy: 0.9705
Epoch 34/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7951 - accuracy: 0.9397 - val_loss: 2.8928 - val_accuracy: 0.8640
Epoch 35/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6916 - accuracy: 0.9418 - val_loss: 0.4184 - val_accuracy: 0.9513
Epoch 36/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6654 - accuracy: 0.9437 - val_loss: 0.2184 - val_accuracy: 0.9684
Epoch 37/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6069 - accuracy: 0.9454 - val_loss: 0.5737 - val_accuracy: 0.9069
Epoch 38/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7128 - accuracy: 0.9420 - val_loss: 0.5033 - val_accuracy: 0.9555
Epoch 39/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7056 - accuracy: 0.9452 - val_loss: 0.2941 - val_accuracy: 0.9691
Epoch 40/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5995 - accuracy: 0.9485 - val_loss: 0.4369 - val_accuracy: 0.9641
Epoch 41/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8250 - accuracy: 0.9419 - val_loss: 0.2247 - val_accuracy: 0.9759
Epoch 42/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6234 - accuracy: 0.9487 - val_loss: 0.7023 - val_accuracy: 0.9447
Epoch 43/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5736 - accuracy: 0.9546 - val_loss: 0.2732 - val_accuracy: 0.9774
Epoch 44/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6399 - accuracy: 0.9505 - val_loss: 0.2177 - val_accuracy: 0.9759
Epoch 45/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6734 - accuracy: 0.9522 - val_loss: 0.8556 - val_accuracy: 0.9381
Epoch 46/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4384 - accuracy: 0.9625 - val_loss: 0.2789 - val_accuracy: 0.9686
Epoch 47/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6232 - accuracy: 0.9506 - val_loss: 0.2446 - val_accuracy: 0.9739
Epoch 48/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5530 - accuracy: 0.9540 - val_loss: 4.1906 - val_accuracy: 0.8169
Epoch 49/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4962 - accuracy: 0.9584 - val_loss: 2.5646 - val_accuracy: 0.9479
Epoch 50/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5574 - accuracy: 0.9570 - val_loss: 0.1406 - val_accuracy: 0.9802
Epoch 51/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7385 - accuracy: 0.9533 - val_loss: 0.1817 - val_accuracy: 0.9788
Epoch 52/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5103 - accuracy: 0.9583 - val_loss: 0.1550 - val_accuracy: 0.9806
Epoch 53/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4009 - accuracy: 0.9637 - val_loss: 0.1303 - val_accuracy: 0.9812
Epoch 54/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4380 - accuracy: 0.9614 - val_loss: 0.6993 - val_accuracy: 0.8994
Epoch 55/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4836 - accuracy: 0.9563 - val_loss: 0.4112 - val_accuracy: 0.9448
Epoch 56/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5270 - accuracy: 0.9558 - val_loss: 0.2097 - val_accuracy: 0.9726
Epoch 57/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4099 - accuracy: 0.9625 - val_loss: 0.3920 - val_accuracy: 0.9617
Epoch 58/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4578 - accuracy: 0.9592 - val_loss: 0.2838 - val_accuracy: 0.9675
Epoch 59/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4295 - accuracy: 0.9633 - val_loss: 0.3050 - val_accuracy: 0.9464
Epoch 60/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3329 - accuracy: 0.9649 - val_loss: 0.6618 - val_accuracy: 0.9145
Epoch 61/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3225 - accuracy: 0.9647 - val_loss: 1.8496 - val_accuracy: 0.8707
Epoch 62/100
560/560 [==============================] - 1s 1ms/step - loss: 0.5362 - accuracy: 0.9534 - val_loss: 0.4548 - val_accuracy: 0.9347
Epoch 63/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3319 - accuracy: 0.9646 - val_loss: 0.9590 - val_accuracy: 0.9205
Epoch 64/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4540 - accuracy: 0.9561 - val_loss: 0.1728 - val_accuracy: 0.9765
Epoch 65/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3413 - accuracy: 0.9632 - val_loss: 0.1453 - val_accuracy: 0.9807
Epoch 66/100
560/560 [==============================] - 1s 1ms/step - loss: 0.2635 - accuracy: 0.9677 - val_loss: 0.1068 - val_accuracy: 0.9820
Epoch 67/100
560/560 [==============================] - 1s 1ms/step - loss: 0.2517 - accuracy: 0.9680 - val_loss: 0.2083 - val_accuracy: 0.9708
Epoch 68/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3018 - accuracy: 0.9620 - val_loss: 0.1046 - val_accuracy: 0.9818
Epoch 69/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3175 - accuracy: 0.9609 - val_loss: 0.1373 - val_accuracy: 0.9746
Epoch 70/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1657 - accuracy: 0.9758 - val_loss: 0.1338 - val_accuracy: 0.9783
Epoch 71/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1824 - accuracy: 0.9716 - val_loss: 0.0746 - val_accuracy: 0.9849
Epoch 72/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3037 - accuracy: 0.9561 - val_loss: 0.0803 - val_accuracy: 0.9817
Epoch 73/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1654 - accuracy: 0.9689 - val_loss: 0.0839 - val_accuracy: 0.9779
Epoch 74/100
560/560 [==============================] - 1s 1ms/step - loss: 0.2489 - accuracy: 0.9588 - val_loss: 0.0938 - val_accuracy: 0.9789
Epoch 75/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1593 - accuracy: 0.9712 - val_loss: 0.1394 - val_accuracy: 0.9609
Epoch 76/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1920 - accuracy: 0.9625 - val_loss: 0.1227 - val_accuracy: 0.9683
Epoch 77/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1744 - accuracy: 0.9654 - val_loss: 0.0642 - val_accuracy: 0.9834
Epoch 78/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1624 - accuracy: 0.9671 - val_loss: 0.0981 - val_accuracy: 0.9706
Epoch 79/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1593 - accuracy: 0.9666 - val_loss: 0.0753 - val_accuracy: 0.9797
Epoch 80/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1171 - accuracy: 0.9744 - val_loss: 0.7171 - val_accuracy: 0.8743
Epoch 81/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1550 - accuracy: 0.9666 - val_loss: 0.1945 - val_accuracy: 0.9405
Epoch 82/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1375 - accuracy: 0.9687 - val_loss: 0.1041 - val_accuracy: 0.9693
Epoch 83/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1435 - accuracy: 0.9666 - val_loss: 0.0844 - val_accuracy: 0.9838
Epoch 84/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1295 - accuracy: 0.9692 - val_loss: 0.0784 - val_accuracy: 0.9746
Epoch 85/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1210 - accuracy: 0.9706 - val_loss: 0.1139 - val_accuracy: 0.9642
Epoch 86/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1389 - accuracy: 0.9671 - val_loss: 0.0560 - val_accuracy: 0.9827
Epoch 87/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0922 - accuracy: 0.9764 - val_loss: 0.0544 - val_accuracy: 0.9858
Epoch 88/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1183 - accuracy: 0.9705 - val_loss: 0.0475 - val_accuracy: 0.9869
Epoch 89/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1082 - accuracy: 0.9737 - val_loss: 0.0812 - val_accuracy: 0.9775
Epoch 90/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1387 - accuracy: 0.9724 - val_loss: 0.0494 - val_accuracy: 0.9851
Epoch 91/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0984 - accuracy: 0.9764 - val_loss: 0.2198 - val_accuracy: 0.9189
Epoch 92/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1179 - accuracy: 0.9715 - val_loss: 0.0945 - val_accuracy: 0.9749
Epoch 93/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0868 - accuracy: 0.9785 - val_loss: 0.1282 - val_accuracy: 0.9587
Epoch 94/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0959 - accuracy: 0.9763 - val_loss: 0.0546 - val_accuracy: 0.9845
Epoch 95/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1367 - accuracy: 0.9690 - val_loss: 0.0648 - val_accuracy: 0.9812
Epoch 96/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0937 - accuracy: 0.9773 - val_loss: 0.0576 - val_accuracy: 0.9858
Epoch 97/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0990 - accuracy: 0.9762 - val_loss: 0.0481 - val_accuracy: 0.9864
Epoch 98/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0835 - accuracy: 0.9791 - val_loss: 0.0476 - val_accuracy: 0.9849
Epoch 99/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0925 - accuracy: 0.9761 - val_loss: 0.0470 - val_accuracy: 0.9861
Epoch 100/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0948 - accuracy: 0.9765 - val_loss: 0.0556 - val_accuracy: 0.9860

Let's fit the sequential model --> model_128_128_64_1
Epoch 1/100
560/560 [==============================] - 2s 2ms/step - loss: 10.8841 - accuracy: 0.7965 - val_loss: 2.4835 - val_accuracy: 0.8477
Epoch 2/100
560/560 [==============================] - 1s 2ms/step - loss: 4.4527 - accuracy: 0.8421 - val_loss: 1.3747 - val_accuracy: 0.9274
Epoch 3/100
560/560 [==============================] - 1s 2ms/step - loss: 1.7109 - accuracy: 0.8789 - val_loss: 1.8130 - val_accuracy: 0.8573
Epoch 4/100
560/560 [==============================] - 1s 2ms/step - loss: 1.7174 - accuracy: 0.8865 - val_loss: 1.9300 - val_accuracy: 0.9130
Epoch 5/100
560/560 [==============================] - 1s 2ms/step - loss: 1.5911 - accuracy: 0.8911 - val_loss: 0.4838 - val_accuracy: 0.9353
Epoch 6/100
560/560 [==============================] - 1s 2ms/step - loss: 1.0907 - accuracy: 0.8968 - val_loss: 1.1536 - val_accuracy: 0.8645
Epoch 7/100
560/560 [==============================] - 1s 2ms/step - loss: 1.0851 - accuracy: 0.8965 - val_loss: 2.3155 - val_accuracy: 0.8625
Epoch 8/100
560/560 [==============================] - 1s 2ms/step - loss: 0.9718 - accuracy: 0.9020 - val_loss: 0.4844 - val_accuracy: 0.9414
Epoch 9/100
560/560 [==============================] - 1s 2ms/step - loss: 0.4100 - accuracy: 0.9210 - val_loss: 0.2865 - val_accuracy: 0.9273
Epoch 10/100
560/560 [==============================] - 1s 2ms/step - loss: 0.5596 - accuracy: 0.9058 - val_loss: 0.5817 - val_accuracy: 0.8769
Epoch 11/100
560/560 [==============================] - 1s 2ms/step - loss: 0.3534 - accuracy: 0.9144 - val_loss: 0.1947 - val_accuracy: 0.9381
Epoch 12/100
560/560 [==============================] - 1s 2ms/step - loss: 0.2383 - accuracy: 0.9283 - val_loss: 0.2178 - val_accuracy: 0.9156
Epoch 13/100
560/560 [==============================] - 1s 2ms/step - loss: 0.2212 - accuracy: 0.9286 - val_loss: 0.2774 - val_accuracy: 0.9271
Epoch 14/100
560/560 [==============================] - 1s 2ms/step - loss: 0.4870 - accuracy: 0.9030 - val_loss: 0.1424 - val_accuracy: 0.9462
Epoch 15/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1654 - accuracy: 0.9385 - val_loss: 0.1347 - val_accuracy: 0.9498
Epoch 16/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1586 - accuracy: 0.9410 - val_loss: 0.1507 - val_accuracy: 0.9423
Epoch 17/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1615 - accuracy: 0.9399 - val_loss: 0.1265 - val_accuracy: 0.9505
Epoch 18/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1587 - accuracy: 0.9401 - val_loss: 0.2107 - val_accuracy: 0.9099
Epoch 19/100
560/560 [==============================] - 1s 2ms/step - loss: 0.2250 - accuracy: 0.9198 - val_loss: 0.1513 - val_accuracy: 0.9403
Epoch 20/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1752 - accuracy: 0.9360 - val_loss: 0.1212 - val_accuracy: 0.9490
Epoch 21/100
560/560 [==============================] - 1s 2ms/step - loss: 0.5648 - accuracy: 0.8907 - val_loss: 0.9665 - val_accuracy: 0.9154
Epoch 22/100
560/560 [==============================] - 1s 2ms/step - loss: 0.3107 - accuracy: 0.9105 - val_loss: 0.4754 - val_accuracy: 0.8538
Epoch 23/100
560/560 [==============================] - 1s 2ms/step - loss: 0.2247 - accuracy: 0.9165 - val_loss: 0.2053 - val_accuracy: 0.9301
Epoch 24/100
560/560 [==============================] - 1s 2ms/step - loss: 0.2092 - accuracy: 0.9226 - val_loss: 0.1647 - val_accuracy: 0.9344
Epoch 25/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1973 - accuracy: 0.9280 - val_loss: 0.1441 - val_accuracy: 0.9395
Epoch 26/100
560/560 [==============================] - 1s 2ms/step - loss: 0.2024 - accuracy: 0.9267 - val_loss: 0.2442 - val_accuracy: 0.9106
Epoch 27/100
560/560 [==============================] - 1s 2ms/step - loss: 0.2031 - accuracy: 0.9260 - val_loss: 0.1336 - val_accuracy: 0.9499
Epoch 28/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1748 - accuracy: 0.9370 - val_loss: 0.2757 - val_accuracy: 0.8829
Epoch 29/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1993 - accuracy: 0.9274 - val_loss: 0.1419 - val_accuracy: 0.9507
Epoch 30/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1622 - accuracy: 0.9393 - val_loss: 0.1805 - val_accuracy: 0.9394
Epoch 31/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1854 - accuracy: 0.9303 - val_loss: 0.1209 - val_accuracy: 0.9522
Epoch 32/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1582 - accuracy: 0.9407 - val_loss: 0.1432 - val_accuracy: 0.9506
Epoch 33/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1589 - accuracy: 0.9408 - val_loss: 0.1136 - val_accuracy: 0.9524
Epoch 34/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1800 - accuracy: 0.9309 - val_loss: 0.1176 - val_accuracy: 0.9534
Epoch 35/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1559 - accuracy: 0.9386 - val_loss: 0.2247 - val_accuracy: 0.9259
Epoch 36/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1363 - accuracy: 0.9466 - val_loss: 0.0945 - val_accuracy: 0.9569
Epoch 37/100
560/560 [==============================] - 1s 2ms/step - loss: 0.3205 - accuracy: 0.8819 - val_loss: 0.3289 - val_accuracy: 0.8563
Epoch 38/100
560/560 [==============================] - 1s 2ms/step - loss: 0.3315 - accuracy: 0.8575 - val_loss: 0.3289 - val_accuracy: 0.8571
Epoch 39/100
560/560 [==============================] - 1s 2ms/step - loss: 0.3270 - accuracy: 0.8587 - val_loss: 0.3245 - val_accuracy: 0.8586
Epoch 40/100
560/560 [==============================] - 1s 2ms/step - loss: 0.3287 - accuracy: 0.8582 - val_loss: 0.3289 - val_accuracy: 0.8567
Epoch 41/100
560/560 [==============================] - 1s 2ms/step - loss: 0.3012 - accuracy: 0.8764 - val_loss: 0.1377 - val_accuracy: 0.9544
Epoch 42/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1474 - accuracy: 0.9458 - val_loss: 0.1408 - val_accuracy: 0.9569
Epoch 43/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1434 - accuracy: 0.9458 - val_loss: 0.0761 - val_accuracy: 0.9689
Epoch 44/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1164 - accuracy: 0.9573 - val_loss: 0.0800 - val_accuracy: 0.9649
Epoch 45/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1200 - accuracy: 0.9552 - val_loss: 0.0819 - val_accuracy: 0.9683
Epoch 46/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0931 - accuracy: 0.9652 - val_loss: 0.0747 - val_accuracy: 0.9692
Epoch 47/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1312 - accuracy: 0.9514 - val_loss: 0.0750 - val_accuracy: 0.9724
Epoch 48/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0901 - accuracy: 0.9663 - val_loss: 0.0633 - val_accuracy: 0.9752
Epoch 49/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0976 - accuracy: 0.9649 - val_loss: 0.1633 - val_accuracy: 0.9730
Epoch 50/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0857 - accuracy: 0.9728 - val_loss: 0.1034 - val_accuracy: 0.9626
Epoch 51/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0737 - accuracy: 0.9734 - val_loss: 0.0623 - val_accuracy: 0.9755
Epoch 52/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0759 - accuracy: 0.9721 - val_loss: 0.0567 - val_accuracy: 0.9783
Epoch 53/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0631 - accuracy: 0.9758 - val_loss: 0.0526 - val_accuracy: 0.9791
Epoch 54/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0619 - accuracy: 0.9754 - val_loss: 0.0476 - val_accuracy: 0.9815
Epoch 55/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0798 - accuracy: 0.9718 - val_loss: 0.0822 - val_accuracy: 0.9715
Epoch 56/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0668 - accuracy: 0.9755 - val_loss: 0.0691 - val_accuracy: 0.9713
Epoch 57/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0601 - accuracy: 0.9780 - val_loss: 0.0546 - val_accuracy: 0.9785
Epoch 58/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0682 - accuracy: 0.9751 - val_loss: 0.0496 - val_accuracy: 0.9796
Epoch 59/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0628 - accuracy: 0.9766 - val_loss: 0.0437 - val_accuracy: 0.9840
Epoch 60/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0604 - accuracy: 0.9774 - val_loss: 0.0434 - val_accuracy: 0.9838
Epoch 61/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0600 - accuracy: 0.9781 - val_loss: 0.0452 - val_accuracy: 0.9807
Epoch 62/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0610 - accuracy: 0.9774 - val_loss: 0.0442 - val_accuracy: 0.9813
Epoch 63/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0593 - accuracy: 0.9778 - val_loss: 0.0610 - val_accuracy: 0.9732
Epoch 64/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0606 - accuracy: 0.9781 - val_loss: 0.1467 - val_accuracy: 0.9588
Epoch 65/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0663 - accuracy: 0.9758 - val_loss: 0.0493 - val_accuracy: 0.9824
Epoch 66/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0590 - accuracy: 0.9777 - val_loss: 0.0564 - val_accuracy: 0.9768
Epoch 67/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0542 - accuracy: 0.9792 - val_loss: 0.0479 - val_accuracy: 0.9800
Epoch 68/100
560/560 [==============================] - 1s 2ms/step - loss: 0.2788 - accuracy: 0.9769 - val_loss: 0.0458 - val_accuracy: 0.9816
Epoch 69/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0574 - accuracy: 0.9791 - val_loss: 0.0873 - val_accuracy: 0.9696
Epoch 70/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0543 - accuracy: 0.9786 - val_loss: 0.0495 - val_accuracy: 0.9785
Epoch 71/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0538 - accuracy: 0.9787 - val_loss: 0.0544 - val_accuracy: 0.9758
Epoch 72/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0566 - accuracy: 0.9781 - val_loss: 0.0419 - val_accuracy: 0.9848
Epoch 73/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0550 - accuracy: 0.9789 - val_loss: 0.0646 - val_accuracy: 0.9716
Epoch 74/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0559 - accuracy: 0.9785 - val_loss: 0.0405 - val_accuracy: 0.9843
Epoch 75/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0531 - accuracy: 0.9797 - val_loss: 0.0408 - val_accuracy: 0.9854
Epoch 76/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0616 - accuracy: 0.9769 - val_loss: 0.0513 - val_accuracy: 0.9791
Epoch 77/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0549 - accuracy: 0.9791 - val_loss: 0.0558 - val_accuracy: 0.9747
Epoch 78/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0497 - accuracy: 0.9808 - val_loss: 0.0403 - val_accuracy: 0.9841
Epoch 79/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0517 - accuracy: 0.9801 - val_loss: 0.0515 - val_accuracy: 0.9797
Epoch 80/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0533 - accuracy: 0.9795 - val_loss: 0.0552 - val_accuracy: 0.9792
Epoch 81/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0496 - accuracy: 0.9811 - val_loss: 0.1039 - val_accuracy: 0.9769
Epoch 82/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1160 - accuracy: 0.9604 - val_loss: 0.0648 - val_accuracy: 0.9784
Epoch 83/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0812 - accuracy: 0.9714 - val_loss: 0.0761 - val_accuracy: 0.9685
Epoch 84/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0665 - accuracy: 0.9767 - val_loss: 0.0590 - val_accuracy: 0.9799
Epoch 85/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0939 - accuracy: 0.9677 - val_loss: 0.0667 - val_accuracy: 0.9708
Epoch 86/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0860 - accuracy: 0.9703 - val_loss: 0.2052 - val_accuracy: 0.9290
Epoch 87/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0602 - accuracy: 0.9793 - val_loss: 0.0534 - val_accuracy: 0.9818
Epoch 88/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0687 - accuracy: 0.9759 - val_loss: 0.0706 - val_accuracy: 0.9796
Epoch 89/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0808 - accuracy: 0.9723 - val_loss: 0.0543 - val_accuracy: 0.9802
Epoch 90/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0714 - accuracy: 0.9750 - val_loss: 0.0497 - val_accuracy: 0.9812
Epoch 91/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0758 - accuracy: 0.9734 - val_loss: 0.0569 - val_accuracy: 0.9817
Epoch 92/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0872 - accuracy: 0.9694 - val_loss: 0.0511 - val_accuracy: 0.9836
Epoch 93/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0590 - accuracy: 0.9800 - val_loss: 0.0533 - val_accuracy: 0.9792
Epoch 94/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0771 - accuracy: 0.9727 - val_loss: 0.0514 - val_accuracy: 0.9800
Epoch 95/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0537 - accuracy: 0.9808 - val_loss: 0.0556 - val_accuracy: 0.9782
Epoch 96/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0590 - accuracy: 0.9785 - val_loss: 0.0474 - val_accuracy: 0.9811
Epoch 97/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0616 - accuracy: 0.9779 - val_loss: 0.0504 - val_accuracy: 0.9815
Epoch 98/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0562 - accuracy: 0.9796 - val_loss: 0.0445 - val_accuracy: 0.9821

Let's fit the sequential model --> model_16_32_32_16_1
Epoch 1/100
560/560 [==============================] - 2s 2ms/step - loss: 19.7954 - accuracy: 0.7909 - val_loss: 3.4777 - val_accuracy: 0.8463
Epoch 2/100
560/560 [==============================] - 1s 1ms/step - loss: 4.1281 - accuracy: 0.8468 - val_loss: 1.6140 - val_accuracy: 0.9110
Epoch 3/100
560/560 [==============================] - 1s 1ms/step - loss: 3.5075 - accuracy: 0.8554 - val_loss: 1.3794 - val_accuracy: 0.8799
Epoch 4/100
560/560 [==============================] - 1s 1ms/step - loss: 1.9756 - accuracy: 0.8709 - val_loss: 0.5407 - val_accuracy: 0.9235
Epoch 5/100
560/560 [==============================] - 1s 1ms/step - loss: 2.0534 - accuracy: 0.8737 - val_loss: 1.3026 - val_accuracy: 0.8777
Epoch 6/100
560/560 [==============================] - 1s 1ms/step - loss: 2.1579 - accuracy: 0.8728 - val_loss: 0.9035 - val_accuracy: 0.9211
Epoch 7/100
560/560 [==============================] - 1s 1ms/step - loss: 1.1602 - accuracy: 0.8823 - val_loss: 1.6485 - val_accuracy: 0.8349
Epoch 8/100
560/560 [==============================] - 1s 1ms/step - loss: 1.0462 - accuracy: 0.8931 - val_loss: 0.5624 - val_accuracy: 0.9068
Epoch 9/100
560/560 [==============================] - 1s 1ms/step - loss: 0.8627 - accuracy: 0.8908 - val_loss: 0.6943 - val_accuracy: 0.9161
Epoch 10/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6690 - accuracy: 0.9018 - val_loss: 0.4000 - val_accuracy: 0.9093
Epoch 11/100
560/560 [==============================] - 1s 1ms/step - loss: 0.6616 - accuracy: 0.9035 - val_loss: 0.2575 - val_accuracy: 0.9473
Epoch 12/100
560/560 [==============================] - 1s 1ms/step - loss: 0.7927 - accuracy: 0.8915 - val_loss: 0.9443 - val_accuracy: 0.8741
Epoch 13/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4964 - accuracy: 0.9086 - val_loss: 0.2268 - val_accuracy: 0.9377
Epoch 14/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4850 - accuracy: 0.9079 - val_loss: 0.3202 - val_accuracy: 0.9310
Epoch 15/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4544 - accuracy: 0.9093 - val_loss: 2.5897 - val_accuracy: 0.7027
Epoch 16/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3738 - accuracy: 0.9177 - val_loss: 0.3057 - val_accuracy: 0.8959
Epoch 17/100
560/560 [==============================] - 1s 1ms/step - loss: 0.4251 - accuracy: 0.9146 - val_loss: 0.1769 - val_accuracy: 0.9481
Epoch 18/100
560/560 [==============================] - 1s 1ms/step - loss: 0.2982 - accuracy: 0.9261 - val_loss: 0.5304 - val_accuracy: 0.8592
Epoch 19/100
560/560 [==============================] - 1s 1ms/step - loss: 0.3043 - accuracy: 0.9227 - val_loss: 0.2052 - val_accuracy: 0.9366
Epoch 20/100
560/560 [==============================] - 1s 1ms/step - loss: 0.2573 - accuracy: 0.9299 - val_loss: 0.1580 - val_accuracy: 0.9439
Epoch 21/100
560/560 [==============================] - 1s 1ms/step - loss: 0.2098 - accuracy: 0.9378 - val_loss: 0.1611 - val_accuracy: 0.9459
Epoch 22/100
560/560 [==============================] - 1s 1ms/step - loss: 0.2148 - accuracy: 0.9359 - val_loss: 0.1191 - val_accuracy: 0.9565
Epoch 23/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1976 - accuracy: 0.9398 - val_loss: 0.3928 - val_accuracy: 0.8705
Epoch 24/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1850 - accuracy: 0.9414 - val_loss: 0.4398 - val_accuracy: 0.8677
Epoch 25/100
560/560 [==============================] - 1s 1ms/step - loss: 0.2156 - accuracy: 0.9451 - val_loss: 0.5899 - val_accuracy: 0.8756
Epoch 26/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1497 - accuracy: 0.9512 - val_loss: 0.1198 - val_accuracy: 0.9550
Epoch 27/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1665 - accuracy: 0.9481 - val_loss: 0.3942 - val_accuracy: 0.8627
Epoch 28/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1275 - accuracy: 0.9565 - val_loss: 0.0706 - val_accuracy: 0.9739
Epoch 29/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1232 - accuracy: 0.9575 - val_loss: 0.1239 - val_accuracy: 0.9587
Epoch 30/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1077 - accuracy: 0.9632 - val_loss: 0.0664 - val_accuracy: 0.9768
Epoch 31/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1164 - accuracy: 0.9607 - val_loss: 0.2258 - val_accuracy: 0.9356
Epoch 32/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1073 - accuracy: 0.9647 - val_loss: 0.1102 - val_accuracy: 0.9563
Epoch 33/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0925 - accuracy: 0.9673 - val_loss: 0.0923 - val_accuracy: 0.9679
Epoch 34/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0885 - accuracy: 0.9702 - val_loss: 0.1158 - val_accuracy: 0.9570
Epoch 35/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0935 - accuracy: 0.9704 - val_loss: 0.1809 - val_accuracy: 0.9214
Epoch 36/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0848 - accuracy: 0.9702 - val_loss: 0.1062 - val_accuracy: 0.9603
Epoch 37/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0984 - accuracy: 0.9696 - val_loss: 0.0922 - val_accuracy: 0.9702
Epoch 38/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0793 - accuracy: 0.9739 - val_loss: 0.1249 - val_accuracy: 0.9619
Epoch 39/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0707 - accuracy: 0.9758 - val_loss: 0.0404 - val_accuracy: 0.9842
Epoch 40/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0757 - accuracy: 0.9741 - val_loss: 0.0488 - val_accuracy: 0.9831
Epoch 41/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0646 - accuracy: 0.9774 - val_loss: 0.0443 - val_accuracy: 0.9825
Epoch 42/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0667 - accuracy: 0.9768 - val_loss: 0.0561 - val_accuracy: 0.9798
Epoch 43/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0633 - accuracy: 0.9786 - val_loss: 0.0337 - val_accuracy: 0.9880
Epoch 44/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0544 - accuracy: 0.9805 - val_loss: 0.0495 - val_accuracy: 0.9794
Epoch 45/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0578 - accuracy: 0.9791 - val_loss: 0.0354 - val_accuracy: 0.9879
Epoch 46/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0636 - accuracy: 0.9781 - val_loss: 0.0396 - val_accuracy: 0.9821
Epoch 47/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0497 - accuracy: 0.9831 - val_loss: 0.1348 - val_accuracy: 0.9511
Epoch 48/100
560/560 [==============================] - 1s 1ms/step - loss: 0.1040 - accuracy: 0.9688 - val_loss: 0.1092 - val_accuracy: 0.9557
Epoch 49/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0608 - accuracy: 0.9790 - val_loss: 0.0843 - val_accuracy: 0.9740
Epoch 50/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0550 - accuracy: 0.9809 - val_loss: 0.0349 - val_accuracy: 0.9887
Epoch 51/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0531 - accuracy: 0.9814 - val_loss: 0.0503 - val_accuracy: 0.9801
Epoch 52/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0634 - accuracy: 0.9788 - val_loss: 0.0459 - val_accuracy: 0.9836
Epoch 53/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0538 - accuracy: 0.9816 - val_loss: 0.0381 - val_accuracy: 0.9849
Epoch 54/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0616 - accuracy: 0.9792 - val_loss: 0.0544 - val_accuracy: 0.9802
Epoch 55/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0485 - accuracy: 0.9828 - val_loss: 0.0572 - val_accuracy: 0.9765
Epoch 56/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0639 - accuracy: 0.9828 - val_loss: 0.0364 - val_accuracy: 0.9861
Epoch 57/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0438 - accuracy: 0.9844 - val_loss: 0.0263 - val_accuracy: 0.9918
Epoch 58/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0492 - accuracy: 0.9836 - val_loss: 0.0790 - val_accuracy: 0.9657
Epoch 59/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0432 - accuracy: 0.9846 - val_loss: 0.0238 - val_accuracy: 0.9934
Epoch 60/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0382 - accuracy: 0.9858 - val_loss: 0.0248 - val_accuracy: 0.9926
Epoch 61/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0529 - accuracy: 0.9827 - val_loss: 0.0270 - val_accuracy: 0.9903
Epoch 62/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0415 - accuracy: 0.9856 - val_loss: 0.0249 - val_accuracy: 0.9920
Epoch 63/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0642 - accuracy: 0.9795 - val_loss: 0.1591 - val_accuracy: 0.9493
Epoch 64/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0904 - accuracy: 0.9709 - val_loss: 0.0897 - val_accuracy: 0.9662
Epoch 65/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0760 - accuracy: 0.9763 - val_loss: 0.0940 - val_accuracy: 0.9716
Epoch 66/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0705 - accuracy: 0.9788 - val_loss: 0.0925 - val_accuracy: 0.9693
Epoch 67/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0699 - accuracy: 0.9793 - val_loss: 0.0603 - val_accuracy: 0.9807
Epoch 68/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0730 - accuracy: 0.9787 - val_loss: 0.0793 - val_accuracy: 0.9750
Epoch 69/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0708 - accuracy: 0.9792 - val_loss: 0.0672 - val_accuracy: 0.9804
Epoch 70/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0724 - accuracy: 0.9778 - val_loss: 0.0521 - val_accuracy: 0.9834
Epoch 71/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0666 - accuracy: 0.9798 - val_loss: 0.0706 - val_accuracy: 0.9797
Epoch 72/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0738 - accuracy: 0.9778 - val_loss: 0.0719 - val_accuracy: 0.9779
Epoch 73/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0664 - accuracy: 0.9800 - val_loss: 0.0510 - val_accuracy: 0.9831
Epoch 74/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0670 - accuracy: 0.9795 - val_loss: 0.0512 - val_accuracy: 0.9834
Epoch 75/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0643 - accuracy: 0.9807 - val_loss: 0.0655 - val_accuracy: 0.9810
Epoch 76/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0687 - accuracy: 0.9790 - val_loss: 0.0692 - val_accuracy: 0.9783
Epoch 77/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0707 - accuracy: 0.9780 - val_loss: 0.0687 - val_accuracy: 0.9796
Epoch 78/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0641 - accuracy: 0.9806 - val_loss: 0.0825 - val_accuracy: 0.9726
Epoch 79/100
560/560 [==============================] - 1s 1ms/step - loss: 0.0652 - accuracy: 0.9795 - val_loss: 0.0697 - val_accuracy: 0.9807

Let's fit the sequential model --> model_32_128_64_32_1
Epoch 1/100
560/560 [==============================] - 2s 2ms/step - loss: 7.7251 - accuracy: 0.7428 - val_loss: 1.4146 - val_accuracy: 0.8193
Epoch 2/100
560/560 [==============================] - 1s 2ms/step - loss: 1.5910 - accuracy: 0.8424 - val_loss: 7.0516 - val_accuracy: 0.5633
Epoch 3/100
560/560 [==============================] - 1s 2ms/step - loss: 1.2733 - accuracy: 0.8612 - val_loss: 0.3245 - val_accuracy: 0.9375
Epoch 4/100
560/560 [==============================] - 1s 2ms/step - loss: 0.9507 - accuracy: 0.8653 - val_loss: 0.4351 - val_accuracy: 0.9302
Epoch 5/100
560/560 [==============================] - 1s 2ms/step - loss: 0.6111 - accuracy: 0.8860 - val_loss: 0.2838 - val_accuracy: 0.9330
Epoch 6/100
560/560 [==============================] - 1s 2ms/step - loss: 0.6609 - accuracy: 0.8835 - val_loss: 0.2347 - val_accuracy: 0.9334
Epoch 7/100
560/560 [==============================] - 1s 2ms/step - loss: 0.4614 - accuracy: 0.8978 - val_loss: 0.2837 - val_accuracy: 0.9150
Epoch 8/100
560/560 [==============================] - 1s 2ms/step - loss: 0.3919 - accuracy: 0.9051 - val_loss: 0.2493 - val_accuracy: 0.9345
Epoch 9/100
560/560 [==============================] - 1s 2ms/step - loss: 0.3697 - accuracy: 0.9056 - val_loss: 0.2059 - val_accuracy: 0.9292
Epoch 10/100
560/560 [==============================] - 1s 2ms/step - loss: 0.3018 - accuracy: 0.9132 - val_loss: 0.1514 - val_accuracy: 0.9441
Epoch 11/100
560/560 [==============================] - 1s 2ms/step - loss: 0.2473 - accuracy: 0.9221 - val_loss: 0.1898 - val_accuracy: 0.9391
Epoch 12/100
560/560 [==============================] - 1s 2ms/step - loss: 0.3941 - accuracy: 0.9082 - val_loss: 1.4710 - val_accuracy: 0.7281
Epoch 13/100
560/560 [==============================] - 1s 2ms/step - loss: 0.2389 - accuracy: 0.9249 - val_loss: 0.1730 - val_accuracy: 0.9417
Epoch 14/100
560/560 [==============================] - 1s 2ms/step - loss: 0.2228 - accuracy: 0.9260 - val_loss: 0.1354 - val_accuracy: 0.9468
Epoch 15/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1978 - accuracy: 0.9302 - val_loss: 0.2019 - val_accuracy: 0.9266
Epoch 16/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1780 - accuracy: 0.9365 - val_loss: 0.3468 - val_accuracy: 0.8566
Epoch 17/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1816 - accuracy: 0.9327 - val_loss: 0.1360 - val_accuracy: 0.9506
Epoch 18/100
560/560 [==============================] - 1s 2ms/step - loss: 0.2790 - accuracy: 0.9283 - val_loss: 0.8369 - val_accuracy: 0.8684
Epoch 19/100
560/560 [==============================] - 1s 2ms/step - loss: 0.2011 - accuracy: 0.9247 - val_loss: 0.1231 - val_accuracy: 0.9507
Epoch 20/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1401 - accuracy: 0.9458 - val_loss: 0.1082 - val_accuracy: 0.9529
Epoch 21/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1422 - accuracy: 0.9462 - val_loss: 0.1185 - val_accuracy: 0.9574
Epoch 22/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1421 - accuracy: 0.9447 - val_loss: 0.1032 - val_accuracy: 0.9545
Epoch 23/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1263 - accuracy: 0.9508 - val_loss: 0.1044 - val_accuracy: 0.9567
Epoch 24/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1187 - accuracy: 0.9534 - val_loss: 0.1392 - val_accuracy: 0.9450
Epoch 25/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1834 - accuracy: 0.9503 - val_loss: 0.2256 - val_accuracy: 0.9371
Epoch 26/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1270 - accuracy: 0.9509 - val_loss: 0.1566 - val_accuracy: 0.9464
Epoch 27/100
560/560 [==============================] - 1s 2ms/step - loss: 0.3177 - accuracy: 0.9473 - val_loss: 0.1117 - val_accuracy: 0.9560
Epoch 28/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1191 - accuracy: 0.9544 - val_loss: 0.1508 - val_accuracy: 0.9523
Epoch 29/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1108 - accuracy: 0.9581 - val_loss: 0.0920 - val_accuracy: 0.9636
Epoch 30/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1066 - accuracy: 0.9591 - val_loss: 0.0745 - val_accuracy: 0.9683
Epoch 31/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0972 - accuracy: 0.9636 - val_loss: 0.1608 - val_accuracy: 0.9484
Epoch 32/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0917 - accuracy: 0.9653 - val_loss: 0.0671 - val_accuracy: 0.9722
Epoch 33/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0837 - accuracy: 0.9684 - val_loss: 0.1441 - val_accuracy: 0.9527
Epoch 34/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0807 - accuracy: 0.9703 - val_loss: 0.0656 - val_accuracy: 0.9757
Epoch 35/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0787 - accuracy: 0.9711 - val_loss: 0.0553 - val_accuracy: 0.9786
Epoch 36/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0764 - accuracy: 0.9724 - val_loss: 0.1917 - val_accuracy: 0.9239
Epoch 37/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0740 - accuracy: 0.9738 - val_loss: 0.1084 - val_accuracy: 0.9598
Epoch 38/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0711 - accuracy: 0.9743 - val_loss: 0.0571 - val_accuracy: 0.9779
Epoch 39/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0687 - accuracy: 0.9758 - val_loss: 0.0463 - val_accuracy: 0.9818
Epoch 40/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0710 - accuracy: 0.9750 - val_loss: 0.0619 - val_accuracy: 0.9770
Epoch 41/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0687 - accuracy: 0.9752 - val_loss: 0.0481 - val_accuracy: 0.9820
Epoch 42/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0556 - accuracy: 0.9797 - val_loss: 0.0578 - val_accuracy: 0.9797
Epoch 43/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0627 - accuracy: 0.9774 - val_loss: 0.0548 - val_accuracy: 0.9797
Epoch 44/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0932 - accuracy: 0.9669 - val_loss: 0.0951 - val_accuracy: 0.9712
Epoch 45/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0843 - accuracy: 0.9719 - val_loss: 0.1412 - val_accuracy: 0.9191
Epoch 46/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0730 - accuracy: 0.9745 - val_loss: 0.0528 - val_accuracy: 0.9819
Epoch 47/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0618 - accuracy: 0.9777 - val_loss: 0.0512 - val_accuracy: 0.9793
Epoch 48/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0565 - accuracy: 0.9789 - val_loss: 0.0803 - val_accuracy: 0.9688
Epoch 49/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0610 - accuracy: 0.9781 - val_loss: 0.0447 - val_accuracy: 0.9831
Epoch 50/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0557 - accuracy: 0.9804 - val_loss: 0.0406 - val_accuracy: 0.9844
Epoch 51/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0592 - accuracy: 0.9800 - val_loss: 0.0533 - val_accuracy: 0.9805
Epoch 52/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0661 - accuracy: 0.9768 - val_loss: 0.0385 - val_accuracy: 0.9878
Epoch 53/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0547 - accuracy: 0.9805 - val_loss: 0.0468 - val_accuracy: 0.9832
Epoch 54/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0544 - accuracy: 0.9801 - val_loss: 0.0401 - val_accuracy: 0.9849
Epoch 55/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0528 - accuracy: 0.9811 - val_loss: 0.0376 - val_accuracy: 0.9863
Epoch 56/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0507 - accuracy: 0.9818 - val_loss: 0.0338 - val_accuracy: 0.9871
Epoch 57/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0751 - accuracy: 0.9743 - val_loss: 0.0570 - val_accuracy: 0.9778
Epoch 58/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0602 - accuracy: 0.9787 - val_loss: 0.0420 - val_accuracy: 0.9853
Epoch 59/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0488 - accuracy: 0.9826 - val_loss: 0.0708 - val_accuracy: 0.9696
Epoch 60/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0473 - accuracy: 0.9820 - val_loss: 0.0488 - val_accuracy: 0.9836
Epoch 61/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0530 - accuracy: 0.9807 - val_loss: 0.0311 - val_accuracy: 0.9883
Epoch 62/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0456 - accuracy: 0.9826 - val_loss: 0.0421 - val_accuracy: 0.9859
Epoch 63/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1011 - accuracy: 0.9823 - val_loss: 0.3352 - val_accuracy: 0.9762
Epoch 64/100
560/560 [==============================] - 1s 2ms/step - loss: 0.1079 - accuracy: 0.9814 - val_loss: 0.0430 - val_accuracy: 0.9872
Epoch 65/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0477 - accuracy: 0.9818 - val_loss: 0.0309 - val_accuracy: 0.9887
Epoch 66/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0406 - accuracy: 0.9851 - val_loss: 0.0523 - val_accuracy: 0.9757
Epoch 67/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0383 - accuracy: 0.9854 - val_loss: 0.0260 - val_accuracy: 0.9911
Epoch 68/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0497 - accuracy: 0.9822 - val_loss: 0.0299 - val_accuracy: 0.9896
Epoch 69/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0378 - accuracy: 0.9859 - val_loss: 0.0430 - val_accuracy: 0.9832
Epoch 70/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0467 - accuracy: 0.9822 - val_loss: 0.0280 - val_accuracy: 0.9873
Epoch 71/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0379 - accuracy: 0.9858 - val_loss: 0.0320 - val_accuracy: 0.9872
Epoch 72/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0486 - accuracy: 0.9817 - val_loss: 0.0906 - val_accuracy: 0.9650
Epoch 73/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0364 - accuracy: 0.9860 - val_loss: 0.0541 - val_accuracy: 0.9756
Epoch 74/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0389 - accuracy: 0.9859 - val_loss: 0.0520 - val_accuracy: 0.9792
Epoch 75/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0385 - accuracy: 0.9861 - val_loss: 0.0319 - val_accuracy: 0.9863
Epoch 76/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0352 - accuracy: 0.9867 - val_loss: 0.0570 - val_accuracy: 0.9792
Epoch 77/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0411 - accuracy: 0.9846 - val_loss: 0.0504 - val_accuracy: 0.9839
Epoch 78/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0358 - accuracy: 0.9859 - val_loss: 0.0348 - val_accuracy: 0.9874
Epoch 79/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0388 - accuracy: 0.9854 - val_loss: 0.0927 - val_accuracy: 0.9645
Epoch 80/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0398 - accuracy: 0.9847 - val_loss: 0.0460 - val_accuracy: 0.9804
Epoch 81/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0433 - accuracy: 0.9843 - val_loss: 0.0224 - val_accuracy: 0.9910
Epoch 82/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0344 - accuracy: 0.9871 - val_loss: 0.0236 - val_accuracy: 0.9930
Epoch 83/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0509 - accuracy: 0.9831 - val_loss: 0.0210 - val_accuracy: 0.9929
Epoch 84/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0304 - accuracy: 0.9884 - val_loss: 0.0485 - val_accuracy: 0.9902
Epoch 85/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0344 - accuracy: 0.9876 - val_loss: 0.0499 - val_accuracy: 0.9783
Epoch 86/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0392 - accuracy: 0.9853 - val_loss: 0.0334 - val_accuracy: 0.9861
Epoch 87/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0299 - accuracy: 0.9884 - val_loss: 0.0414 - val_accuracy: 0.9822
Epoch 88/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0465 - accuracy: 0.9826 - val_loss: 0.0327 - val_accuracy: 0.9864
Epoch 89/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0434 - accuracy: 0.9842 - val_loss: 0.0267 - val_accuracy: 0.9903
Epoch 90/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0320 - accuracy: 0.9878 - val_loss: 0.0425 - val_accuracy: 0.9864
Epoch 91/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0364 - accuracy: 0.9866 - val_loss: 0.0275 - val_accuracy: 0.9896
Epoch 92/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0343 - accuracy: 0.9873 - val_loss: 0.0207 - val_accuracy: 0.9927
Epoch 93/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0334 - accuracy: 0.9878 - val_loss: 0.5595 - val_accuracy: 0.9917
Epoch 94/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0728 - accuracy: 0.9885 - val_loss: 0.0263 - val_accuracy: 0.9907
Epoch 95/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0299 - accuracy: 0.9887 - val_loss: 0.0167 - val_accuracy: 0.9942
Epoch 96/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0317 - accuracy: 0.9876 - val_loss: 0.0232 - val_accuracy: 0.9919
Epoch 97/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0348 - accuracy: 0.9870 - val_loss: 0.0156 - val_accuracy: 0.9954
Epoch 98/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0286 - accuracy: 0.9893 - val_loss: 0.0177 - val_accuracy: 0.9936
Epoch 99/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0295 - accuracy: 0.9887 - val_loss: 0.0999 - val_accuracy: 0.9721
Epoch 100/100
560/560 [==============================] - 1s 2ms/step - loss: 0.0312 - accuracy: 0.9884 - val_loss: 0.0164 - val_accuracy: 0.9954

from sklearn import metrics

# Function which evaluates on X_valid and y_valid and compares 
# the results with model checkpoint generated results


def model_evaluation(X_validation, y_validation, model, callback_file):
    # Let's get the loss and accuracy corresponding to particular model
    scores = model.evaluate(X_validation, y_validation, verbose=0)
    print(f"{model.metrics_names[0]}: {scores[0]}")
    print(f"{model.metrics_names[1]}: {scores[1]*100}")

    # Let's load the checkpoint and validate the metrics with above evaluated results
    model.load_weights(callback_file)
    prediction = model.predict(X_validation)
    prediction = (prediction > 0.5)
    confusion_matrix = metrics.confusion_matrix(y_validation, prediction)
    accuracy = (confusion_matrix[0, 0] + confusion_matrix[1, 1]) / sum(confusion_matrix.flatten()) * 100
    print(f"\n Confusion matrix with model evaluate \n{confusion_matrix}")
    print(f"Accuracy score with model evaluate: {accuracy}\n")

    accuracy = metrics.accuracy_score(y_validation, prediction.round()) * 100.0
    precision = metrics.precision_score(y_validation, prediction.round()) * 100.0
    recall = metrics.recall_score(y_validation, prediction.round()) * 100.0
    f1_score = metrics.f1_score(y_validation, prediction.round())

    print(f"Accuracy with model checkpoint: {accuracy}")
    print(f"Precision with model checkpoint: {precision}")
    print(f"Recall with model checkpoint: {recall}")
    print(f"F1-score with model checkpoint: {f1_score}")

    for i in range(5):
        print()
        print(f"Prediction={prediction[i]}")
    return prediction
# Let's initialize a list to store all the predictions corresponding to all the model architectures
adam_predictions = []

for index, (model_name, model) in enumerate(zip(adam_model_names, adam_models)):
    callback_file = f"models/adam_{model_name}.hdf5"
    print(f"===================================== {model_name} ===========================================")
    prediction = model_evaluation(test_inputs, test_outputs, model, callback_file)
    adam_predictions.append(prediction)
    print()
===================================== model_base ===========================================
loss: 0.916689932346344
accuracy: 82.14365243911743
560/560 [==============================] - 0s 650us/step

 Confusion matrix with model evaluate 
[[7731 1221]
 [ 961 7991]]
Accuracy score with model evaluate: 87.81277926720286

Accuracy with model checkpoint: 87.81277926720286
Precision with model checkpoint: 86.7455492835432
Recall with model checkpoint: 89.26496872207328
F1-score with model checkpoint: 0.8798722748293326

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

===================================== model_8_1 ===========================================
loss: 0.41907256841659546
accuracy: 94.10746097564697
560/560 [==============================] - 0s 716us/step

 Confusion matrix with model evaluate 
[[8351  601]
 [ 708 8244]]
Accuracy score with model evaluate: 92.68878462913315

Accuracy with model checkpoint: 92.68878462913315
Precision with model checkpoint: 93.20520067834936
Recall with model checkpoint: 92.0911528150134
F1-score with model checkpoint: 0.9264482777996292

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

===================================== model_4_4_1 ===========================================
loss: 0.6913169026374817
accuracy: 50.262510776519775
560/560 [==============================] - 0s 662us/step

 Confusion matrix with model evaluate 
[[8951    1]
 [8904   48]]
Accuracy score with model evaluate: 50.262511170688114

Accuracy with model checkpoint: 50.262511170688114
Precision with model checkpoint: 97.95918367346938
Recall with model checkpoint: 0.5361930294906166
F1-score with model checkpoint: 0.010665481613154094

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

===================================== model_16_8_1 ===========================================
loss: 0.38308969140052795
accuracy: 93.16353797912598
560/560 [==============================] - 0s 725us/step

 Confusion matrix with model evaluate 
[[8819  133]
 [ 175 8777]]
Accuracy score with model evaluate: 98.27971403038427

Accuracy with model checkpoint: 98.27971403038427
Precision with model checkpoint: 98.50729517396184
Recall with model checkpoint: 98.04512957998213
F1-score with model checkpoint: 0.9827566901802709

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

===================================== model_32_16_1 ===========================================
loss: 0.055607303977012634
accuracy: 98.60366582870483
560/560 [==============================] - 0s 711us/step

 Confusion matrix with model evaluate 
[[8861   91]
 [ 157 8795]]
Accuracy score with model evaluate: 98.6148346738159

Accuracy with model checkpoint: 98.6148346738159
Precision with model checkpoint: 98.97591717308126
Recall with model checkpoint: 98.24620196604111
F1-score with model checkpoint: 0.9860970960870054

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

===================================== model_128_128_64_1 ===========================================
loss: 0.04445268586277962
accuracy: 98.2071042060852
560/560 [==============================] - 0s 745us/step

 Confusion matrix with model evaluate 
[[8783  169]
 [ 115 8837]]
Accuracy score with model evaluate: 98.41376228775692

Accuracy with model checkpoint: 98.41376228775692
Precision with model checkpoint: 98.12347324006218
Recall with model checkpoint: 98.7153708668454
F1-score with model checkpoint: 0.9841853213052678

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

===================================== model_16_32_32_16_1 ===========================================
loss: 0.06969509273767471
accuracy: 98.0674684047699
560/560 [==============================] - 0s 725us/step

 Confusion matrix with model evaluate 
[[8914   38]
 [  81 8871]]
Accuracy score with model evaluate: 99.33534405719392

Accuracy with model checkpoint: 99.33534405719392
Precision with model checkpoint: 99.5734650353575
Recall with model checkpoint: 99.09517426273459
F1-score with model checkpoint: 0.9933374391131515

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

===================================== model_32_128_64_32_1 ===========================================
loss: 0.016375038772821426
accuracy: 99.53641891479492
560/560 [==============================] - 0s 772us/step

 Confusion matrix with model evaluate 
[[8909   43]
 [  40 8912]]
Accuracy score with model evaluate: 99.5364164432529

Accuracy with model checkpoint: 99.5364164432529
Precision with model checkpoint: 99.51982132886656
Recall with model checkpoint: 99.55317247542449
F1-score with model checkpoint: 0.9953649410844921

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]

Prediction=[False]


# Function to plot the validation loss and validation accuracy
def classification_plots(model_name, model_history):
    # Let's create plot with multiple subplots(2 for each row)
    figure, (ax1, ax2) = plt.subplots(1, 2, figsize=(18, 6))
    figure.suptitle(model_name)
    ax1.plot(model_history.history["loss"])
    ax1.plot(model_history.history["val_loss"])
    ax1.set_ylabel("loss")
    ax1.set_xlabel("epoch")
    ax1.legend(["Training loss data", "validation loss data"], loc="best")

    ax2.plot(model_history.history["accuracy"])
    ax2.plot(model_history.history["val_accuracy"])
    ax2.set_ylabel("accuracy")
    ax2.set_xlabel("epoch")
    ax2.legend(["Training accuracy data", "validation accuracy data"], loc="best")
    figure.show()
# Let's iterate over the model names to plot the loss and accuracy
for index, (model_name, model_history) in enumerate(zip(adam_model_names, adam_history)):
    # Let's call the classification_plots 
    classification_plots(model_name=model_name, model_history=model_history)



What are ROC curves? The ROC curve is a graphical representation of the performance of a binary classifier system as the discrimination threshold is varied. It plots the true positive rate against the false positive rate. The area under the curve (AUC) is a performance metric for binary classification problems. The higher the AUC, the better the model is at distinguishing between positive and negative classes.

# Function to plot the ROC curves
def roc_auc_curve(y_validation, y_hypothesis, name="ROC Curve"):
    fpr, tpr, thresholds = metrics.roc_curve(y_validation, y_hypothesis)
    figure, ax = plt.subplots(figsize=(5, 3))
    figure.suptitle(name)
    ax.plot([0, 1], [0, 1], linestyle="dashed", color="b")
    ax.plot(fpr, tpr, color="red", label=("Area under the curve: ", round(metrics.auc(fpr, tpr), 4)))
    ax.set_xlabel("False Positive Rate", fontsize=10)
    ax.set_ylabel("True Positive Rate", fontsize=10)
    ax.legend(loc="best", fontsize=10)
    figure.show()
for (model_name, y_hypothesis) in zip(adam_model_names, adam_predictions):
    roc_auc_curve(
        y_validation=test_outputs,
        y_hypothesis=y_hypothesis,
        name=f"ROC curve of {model_name}"
    )

Verification of predictions using custom prediction function

# Activation functions
# Rectified Linear function
def rectified(x):
    return np.maximum(0, x)


# Sigmoid function
def sigmoid(x):
    return 1 / (1 + np.exp(np.negative(x)))
# Custom predict function to validate the predictions

def custom_predict(display_summary=True, display_weight=True):
    prediction = test_inputs
    total_layers = len(model_32_128_64_32_1.layers)

    if display_summary is not None:
        print(model_32_128_64_32_1.summary())

    for layer_number, layer in enumerate(model_32_128_64_32_1.layers):
        weights = layer.get_weights()[0].T
        biases = layer.get_weights()[1].T

        if display_weight:
            print(f"Layer Number --> {layer_number}")
            print(f"Weights:\n {weights[0]} ...")
            print(f"Bias:\n {biases[:5]}\n")

        holder = []
        for row in prediction:
            value = []
            for i, w in enumerate(weights):
                val = np.dot(w, row) + biases[i]
                value.append(val)

            if layer_number < total_layers - 1:
                holder.append(rectified(value))
            else:
                holder.append(sigmoid(value))

        prediction = np.array(holder)
    prediction = (prediction > 0.5)

    accuracy = metrics.accuracy_score(test_outputs, prediction.round()) * 100.0
    precision = metrics.precision_score(test_outputs, prediction.round()) * 100.0
    recall = metrics.recall_score(test_outputs, prediction.round()) * 100.0
    f1score = metrics.f1_score(test_outputs, prediction.round())

    print("Accuracy: %.2f%%" % (accuracy))
    print("Precision: %.2f%%" % (precision))
    print("Recall: %.2f%%" % (recall))
    print("F1-score: %.2f\n" % (f1score))

    for i in range(10):
        print(f"X={test_inputs[i][:5]}...., Predicted={prediction[i]}")
    return prediction
# Let's call the custom predict function and validate the results
predictions = custom_predict()
Model: "sequential_68"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_229 (Dense)           (None, 32)                416       
                                                                 
 dense_230 (Dense)           (None, 128)               4224      
                                                                 
 dense_231 (Dense)           (None, 64)                8256      
                                                                 
 dense_232 (Dense)           (None, 32)                2080      
                                                                 
 dense_233 (Dense)           (None, 1)                 33        
                                                                 
=================================================================
Total params: 15,009
Trainable params: 15,009
Non-trainable params: 0
_________________________________________________________________
None
Layer Number --> 0
Weights:
 [-0.12409662  0.12722455 -0.12134153 -0.25718355 -0.2586274  -0.18120514
 -0.25432748  0.1216802  -0.07535347  0.3348318   0.24980044  0.16493839] ...
Bias:
 [ 0.00256717  0.07210213 -0.2727735  -0.01247012 -0.10386065]

Layer Number --> 1
Weights:
 [-0.09873178 -0.13047263  0.02772429 -0.16408151 -0.14625902 -0.13343318
 -0.11239237 -0.14515688  0.09377786 -0.1268853   0.02677506 -0.19920957
  0.00733965  0.16095567 -0.04786922  0.10809107 -0.14534615  0.05348032
  0.1812925  -0.06028208  0.00453761  0.02801764 -0.13240477 -0.14584914
  0.05576867 -0.05762722  0.03988228  0.0377162  -0.15576056  0.05077861
 -0.05374363 -0.04987572] ...
Bias:
 [-0.0390458  -0.01391329 -0.05349183 -0.162648    0.00595188]

Layer Number --> 2
Weights:
 [ 9.82152447e-02 -1.46078378e-01  2.36565724e-01  1.20295011e-01
 -4.65326011e-01  1.28362685e-01 -1.92755312e-01 -6.21341020e-02
  6.22372776e-02  1.63978636e-01  1.12310812e-01 -1.25982627e-01
 -4.44233656e-01  1.07281789e-01  1.22137308e-01 -1.11505175e-02
 -5.35451481e-03  1.38286039e-01 -5.05052060e-02 -1.99512988e-02
 -3.91923711e-02  9.25257504e-02 -1.31678447e-01 -1.65824480e-02
  9.27291065e-02  1.15136333e-01  3.58841151e-01  8.81880894e-02
  1.03527091e-01  2.15866294e-05  3.89339589e-03 -1.52412608e-01
 -1.55493855e-01 -8.49988088e-02 -9.23319384e-02 -1.57691464e-01
 -1.06946893e-01 -1.37219653e-01  1.02819264e-01 -1.85936600e-01
 -2.79583372e-02 -6.34177029e-02  9.45184678e-02 -1.13397785e-01
 -6.05864972e-02  4.23725285e-02  2.92876177e-02  5.15544713e-02
 -1.01677649e-01  6.82677478e-02 -2.44919583e-01 -1.44984484e-01
  7.06463680e-02  1.36416927e-01  9.81185809e-02  1.80226937e-02
 -1.28826767e-01  1.14862829e-01  4.23279665e-02  8.73926803e-02
 -1.50078669e-01  1.17532186e-01 -4.14088257e-02 -1.09077752e-01
 -1.35138988e-01  1.05966017e-01 -8.48357379e-03 -1.10674508e-01
 -4.50297333e-02 -5.43096252e-02 -4.17489968e-02 -1.08684897e-01
 -5.47682904e-02  4.74042892e-02 -8.21316242e-03  1.84692368e-02
  1.31113485e-01 -1.40298977e-01 -1.61597412e-03  7.58091435e-02
 -3.79336923e-02  2.40592696e-02 -1.24349959e-01  1.53434211e-02
 -7.76613802e-02  1.12640709e-01  1.00992627e-01  3.45244221e-02
  1.13935389e-01  1.49349168e-01 -8.29593614e-02 -5.12506068e-02
 -9.71420333e-02 -4.32185940e-02 -1.53385445e-01 -1.00214640e-02
 -1.31186709e-01 -7.62559623e-02  2.13216692e-02 -1.25020787e-01
  1.90557003e-01 -1.48786694e-01 -1.50270626e-01  1.22852020e-01
  8.78310204e-02 -7.82178864e-02  2.12341873e-03  1.44744977e-01
  1.44391403e-01 -7.84581602e-02  1.06501639e-01 -2.58677378e-02
  9.15500149e-02  2.55938489e-02 -2.05153450e-01  6.41002581e-02
  9.16939005e-02 -3.24735371e-03 -1.20319061e-01  3.14137861e-02
  7.44178817e-02  2.78219134e-02 -1.44908011e-01 -1.04679026e-01
  8.46598446e-02 -7.45384842e-02 -1.32850483e-01 -1.31884500e-01] ...
Bias:
 [-0.20405191 -0.0267978  -0.25667953 -0.16374603 -0.17114462]

Layer Number --> 3
Weights:
 [ 0.15927091 -0.16950293 -0.05429655 -0.09360539 -0.22702247  0.10236978
 -0.21904017 -0.07035413 -0.12078913  0.21162228  0.11908953 -0.2114995
 -0.16060205  0.023532    0.10950609  0.15255268 -0.01700437  0.16070183
 -0.05210024  0.01210749 -0.17271185 -0.23864838  0.07547133  0.01191165
 -0.00127998  0.04867005 -0.15503396 -0.07355109 -0.00268257 -0.14567253
 -0.08307131 -0.04005537  0.12022589 -0.0686289  -0.10175524  0.03502015
  0.04296172 -0.07819361  0.08621016 -0.11084542 -0.20605269 -0.00880088
  0.12139378 -0.11863571 -0.09208223 -0.10006267 -0.35197484 -0.14065294
 -0.19042525 -0.2456212  -0.23441046 -0.00891862  0.16862684 -0.10142565
 -0.21157628 -0.01808592 -0.2354752  -0.07329814 -0.07178724 -0.15642157
  0.05884042 -0.11532351 -0.00336165 -0.24979323] ...
Bias:
 [ 0.15967673  0.18653284  0.19304134 -0.13501105 -0.40442696]

Layer Number --> 4
Weights:
 [-0.29473475  0.07913276  0.00307398 -0.07839473 -0.08275532  0.10069656
 -0.09096006 -0.06286191  0.0379896  -0.05323452  0.17839319 -0.22657318
  0.1975725   0.07002454 -0.05628942 -0.0291888  -0.05615472  0.12994742
 -0.09644254 -0.00370176  0.06830718 -0.0347584  -0.05894911  0.23462515
 -0.00712107 -0.1991524  -0.1201409  -0.02068955 -0.34488884  0.0283321
 -0.14284343  0.05821932] ...
Bias:
 [0.33427536]


overflow encountered in exp

Accuracy: 99.54%
Precision: 99.52%
Recall: 99.55%
F1-score: 1.00

X=[   25.45    44.79     0.     400.   13222.  ]...., Predicted=[False]
X=[8.56397163e+00 1.61402080e+01 6.59003365e+02 4.00000000e+02
 1.37349954e+04]...., Predicted=[False]
X=[   27.87959485    45.0549372     71.90914055   400.
 12795.8182811 ]...., Predicted=[False]
X=[   45.46493366    16.37478966   822.85417847   400.
 13684.12507079]...., Predicted=[False]
X=[   26.23076881    47.45418451   190.23558482   437.49038988
 12765.50961012]...., Predicted=[False]
X=[-5.51026564e+00  4.36164182e+01  7.57986975e+01  4.00000000e+02
  1.27945974e+04]...., Predicted=[False]
X=[   26.20646499    46.96943577   120.3535012    413.70564234
 12777.82392943]...., Predicted=[False]
X=[   26.78712919    39.34513637    31.05585859   400.
 12840.        ]...., Predicted=[False]
X=[   23.84046775    49.7929367     49.05438472   400.
 13122.97280764]...., Predicted=[False]
X=[1.1747e+01 5.0610e+01 1.6700e+02 4.0000e+02 1.3145e+04]...., Predicted=[False]



Phase - 4 Feature importance and reduction

# Function to calculate the importance of each feature 
# and sorts them accordingly to find out the best fit features

def feature_importance():
    # Let's initialize the list to store the feature wise accuracy
    feature_accuracy = {}

    for index in range(train_inputs.shape[1]):
        # Let's get the values corresponding to the each feature
        train_input_feature = train_inputs[:, index]
        valid_input_feature = test_inputs[:, index]

        model_significance = Sequential()
        model_significance.add(Dense(32, input_dim=1, activation="relu"))
        model_significance.add(Dense(128, input_dim=1, activation="relu"))
        model_significance.add(Dense(64, input_dim=1, activation="relu"))
        model_significance.add(Dense(32, input_dim=1, activation="relu"))
        model_significance.add(Dense(1, activation="sigmoid"))

        # Let's build the model using binary_crossentropy as the loss function 
        # and accuracy as the evaluation metric during the compilation process
        model_significance.compile(loss="binary_crossentropy", optimizer = "adam", metrics=["accuracy"])

        callback_a = ModelCheckpoint(
            filepath = "models/feature_importance.hdf5", 
            monitor="val_loss", 
            save_best_only=True, 
            save_weights_only=True, 
            verbose=0
        )
        callback_b = EarlyStopping(
            monitor="val_loss", 
            mode="min", 
            patience=20, 
            verbose=0
        )

        print(f"Let's fit the sequential model on {data_frame.columns[index]}")
        # Let's fit the sequential model with input features and output label
        history = model_significance.fit(
            x=train_input_feature, 
            y=train_outputs, 
            validation_data=(valid_input_feature, test_outputs), 
            epochs=150, 
            batch_size=100, 
            callbacks=[callback_a, callback_b], 
            verbose=0
        )
        # Let's precit on the validation inputs
        hypothesis = model_significance.predict(valid_input_feature, verbose=0)
        hypothesis = (hypothesis > 0.5)
        accuracy_score = metrics.accuracy_score(test_outputs, hypothesis)
        feature_accuracy[data_frame.columns[index]] = accuracy_score

        print(f"Accuracy score corresponding to {data_frame.columns[index]} --> {accuracy_score}")
        print()
    return feature_accuracy
# Let's call the feature_importance function to calulate the importance of each feature
feature_accuracy = feature_importance()

# Let's print the feature_accuracy dictionary
feature_accuracy
Let's fit the sequential model on Temperature[C]
Accuracy score corresponding to Temperature[C] --> 0.6809651474530831

Let's fit the sequential model on Humidity[%]
Accuracy score corresponding to Humidity[%] --> 0.7572609472743521

Let's fit the sequential model on TVOC[ppb]
Accuracy score corresponding to TVOC[ppb] --> 0.9010277033065237

Let's fit the sequential model on eCO2[ppm]
Accuracy score corresponding to eCO2[ppm] --> 0.5

Let's fit the sequential model on Raw H2
Accuracy score corresponding to Raw H2 --> 0.5

Let's fit the sequential model on Raw Ethanol
Accuracy score corresponding to Raw Ethanol --> 0.5

Let's fit the sequential model on Pressure[hPa]
Accuracy score corresponding to Pressure[hPa] --> 0.5

Let's fit the sequential model on PM1.0
Accuracy score corresponding to PM1.0 --> 0.6963248436103664

Let's fit the sequential model on PM2.5
Accuracy score corresponding to PM2.5 --> 0.6910187667560321

Let's fit the sequential model on NC0.5
Accuracy score corresponding to NC0.5 --> 0.6972743521000894

Let's fit the sequential model on NC1.0
Accuracy score corresponding to NC1.0 --> 0.6934763181411975

Let's fit the sequential model on NC2.5
Accuracy score corresponding to NC2.5 --> 0.6854334226988382


{'Temperature[C]': 0.6809651474530831,
 'Humidity[%]': 0.7572609472743521,
 'TVOC[ppb]': 0.9010277033065237,
 'eCO2[ppm]': 0.5,
 'Raw H2': 0.5,
 'Raw Ethanol': 0.5,
 'Pressure[hPa]': 0.5,
 'PM1.0': 0.6963248436103664,
 'PM2.5': 0.6910187667560321,
 'NC0.5': 0.6972743521000894,
 'NC1.0': 0.6934763181411975,
 'NC2.5': 0.6854334226988382}
# Let's plot the barchart of different features on
# X-axis and corresponding accuracies on y-axis

sorted_feature_accuracy = {key: value for key, value in sorted(feature_accuracy.items(), key=lambda item: item[1])}
plt.figure(figsize=(20, 7))

# Let's convert the acuuracies into percentages
feature_acc = np.array(list(sorted_feature_accuracy.values())) * 100

sns.barplot(x=list(sorted_feature_accuracy.keys()), y=feature_acc, palette="hls")
plt.savefig("plots/feature_importance.png")
plt.show()

Observations

  1. The smoke detection dataset shows that the "TVOC[ppb]" is the most significant characteristic, and this feature results in a validation accuracy of approximately 0.9029.
  2. The smoke detection dataset shows that the "Humidity[%]" and "Raw Ethanol" are the second most significant feature, and this feature results in a validation accuracy of approximately 0.75.
  3. The smoke detection dataset shows that the "Pressure[hPa]", "eCO2[ppm]" and "Raw H2" are the least significant features, and these result in a validation accuracy of approximately 0.50



Performance trend after removing less significant features

# Let's create the dataframes corresponding to training and validation data
train_dataframe = pd.DataFrame(train_inputs)
valid_dataframe = pd.DataFrame(test_inputs)

# Let's take a quick look at the shape of the train dataframe
print("Train dataframe shape -->", train_dataframe.shape)
print()

# Let's take a quick look at the shape of the valid dataframe
print("Validation dataframe shape -->", valid_dataframe.shape)
print()
Train dataframe shape --> (71610, 12)

Validation dataframe shape --> (17904, 12)


# Let's define the logic to exclude the least important features sequentially
feature_significance = {}

features_list = list(sorted_feature_accuracy.keys())
dataframe_columns = data_frame.columns.tolist()

# Let's iterate over the features and exclude the data corresponding 
# to least important feature and train the model
for index, feature_name in enumerate(features_list):
    model_significance_dict = {}
    if not index == len(sorted_feature_accuracy) - 1:
        least_imp_feature_index = dataframe_columns.index(feature_name)

        train_dataframe.drop(least_imp_feature_index, axis=1, inplace=True)
        valid_dataframe.drop(least_imp_feature_index, axis=1, inplace=True)

    model_significance = Sequential()
    model_significance.add(Dense(32, input_dim=train_dataframe.shape[1], activation="relu"))
    model_significance.add(Dense(128, input_dim=train_dataframe.shape[1], activation="relu"))
    model_significance.add(Dense(64, input_dim=train_dataframe.shape[1], activation="relu"))
    model_significance.add(Dense(32, input_dim=train_dataframe.shape[1], activation="relu"))
    model_significance.add(Dense(1, activation="sigmoid"))

    # Let's build the model using binary_crossentropy as the loss function 
    # and accuracy as the evaluation metric during the compilation process
    model_significance.compile(loss="binary_crossentropy", optimizer = "adam", metrics=["accuracy"])

    callback_a = ModelCheckpoint(
        filepath="models/feature_removal.hdf5",
        monitor="val_loss",
        save_best_only=True,
        save_weights_only=True,
        verbose=0
    )
    callback_b = EarlyStopping(
        monitor="val_loss",
        mode="min",
        patience=20,
        verbose=0
    )
    print(f"Let's fit the sequential model excluding {data_frame.columns[least_imp_feature_index]}")
    # Let's fit the sequential model with input features and output label
    history = model_significance.fit(
        x=train_dataframe.values,
        y=train_outputs,
        validation_data=(valid_dataframe.values, test_outputs),
        epochs=150,
        batch_size=100,
        callbacks=[callback_a, callback_b],
        verbose=0
    )

    # Let's precit on the validation inputs
    hypothesis = model_significance.predict(valid_dataframe.values, verbose=0)

    hypothesis = (hypothesis > 0.5)
    accuracy_score = metrics.accuracy_score(test_outputs, hypothesis)
    # model_significance_dict["history"] = history
    feature_significance[f"After removing {data_frame.columns[least_imp_feature_index]}"] = accuracy_score

    print(f"Accuracy score excluding {data_frame.columns[least_imp_feature_index]} --> {accuracy_score}")
    print()
Let's fit the sequential model excluding eCO2[ppm]
Accuracy score excluding eCO2[ppm] --> 0.9947497765862378

Let's fit the sequential model excluding Raw H2
Accuracy score excluding Raw H2 --> 0.9901697944593387

Let's fit the sequential model excluding Raw Ethanol
Accuracy score excluding Raw Ethanol --> 0.9974865951742627

Let's fit the sequential model excluding Pressure[hPa]
Accuracy score excluding Pressure[hPa] --> 0.9934651474530831

Let's fit the sequential model excluding Temperature[C]
Accuracy score excluding Temperature[C] --> 0.9869302949061662

Let's fit the sequential model excluding NC2.5
Accuracy score excluding NC2.5 --> 0.967828418230563

Let's fit the sequential model excluding PM2.5
Accuracy score excluding PM2.5 --> 0.9847520107238605

Let's fit the sequential model excluding NC1.0
Accuracy score excluding NC1.0 --> 0.9840817694369973

Let's fit the sequential model excluding PM1.0
Accuracy score excluding PM1.0 --> 0.9847520107238605

Let's fit the sequential model excluding NC0.5
Accuracy score excluding NC0.5 --> 0.9391756032171582

Let's fit the sequential model excluding Humidity[%]
Accuracy score excluding Humidity[%] --> 0.9029267202859697

Let's fit the sequential model excluding Humidity[%]
Accuracy score excluding Humidity[%] --> 0.9022564789991063


# Let's plot the barchart of different features on
# X-axis and corresponding accuracies on y-axis

sorted_feature_accuracy = {key: value for key, value in sorted(feature_significance.items(), key=lambda item: item[1], reverse=True)}
figure, ax = plt.subplots(figsize=(20, 7))

# Let's convert the acuuracies into percentages
feature_acc = np.array(list(sorted_feature_accuracy.values())) * 100

sns.barplot(x=list(sorted_feature_accuracy.keys()), y=feature_acc, palette="hls")
ax.set_xticklabels(ax.get_xticklabels(), rotation=90)
plt.savefig("plots/feature_removal.png")
plt.show()



Bonus Points Activity



LIME Implementation

# Let's suppress all warnings from lime_tabular
warnings.filterwarnings('ignore', module='lime.lime_tabular')
# Let's initialize a new model instance with best accuracy architecture
model_feature_importance = Sequential()
model_feature_importance.add(Dense(32, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_feature_importance.add(Dense(128, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_feature_importance.add(Dense(64, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_feature_importance.add(Dense(32, input_dim=data_frame.shape[1] - 1, activation="relu"))
model_feature_importance.add(Dense(1, activation="sigmoid"))

model_feature_importance.compile(
    loss="binary_crossentropy",
    optimizer="adam",
    metrics=["accuracy"]
)

history = model_feature_importance.fit(
    x=train_inputs,
    y=train_outputs,
    validation_data=(test_inputs, test_outputs),
    epochs=100,
    batch_size=128,
    verbose=0
)
# Let's define a predict function to generate a prediction for single datapoint
def predict_fn(x):
    feature_predictions = model_feature_importance.predict(x, verbose=0)
    return feature_predictions
# Let's instantiate the LimeTabularExplainer class with the necessary arguments
explainer = LimeTabularExplainer(
    train_inputs,
    mode="classification",
    feature_names=list(data_frame.columns)[:-1],
    class_names=[0, 1],
    discretize_continuous=False,
    verbose=False
)
# Let's initialize an empty list to store all the explanations
explanations = []

# Let's loop through 5000 random datapoints and generate explanations
for i in range(len(test_inputs[:5000])):
    explainer_object = explainer.explain_instance(
        test_inputs[i],
        predict_fn,
        num_features=train_inputs.shape[0],
        top_labels=1
    )
    explanations.append(explainer_object)
# Let's review the explanations of all the features that correspond to three data points
for index, lime_exp_obj in enumerate(explanations[:3], start=1):
    print(f"================Data Point {index}=================")
    lime_exp_obj.show_in_notebook()
================Data Point 1=================

================Data Point 2=================

================Data Point 3=================

# Let's create a bar chart for each feature
for i, feature_name in enumerate(dataframe_columns[:-1]):
    plt.figure(figsize=(15, 8))
    feature_values = np.array([exp.as_list(0)[i][1] for exp in explanations])
    plt.bar(range(len(feature_values[:50])), feature_values[:50])
    plt.xlabel("Datapoint")
    plt.ylabel("Lime feature value")
    plt.title(feature_name)
    plt.show()



Shapley Implementation

# Let's initialize a new model instance with best accuracy architecture

shapley_feature_importance = Sequential()
shapley_feature_importance.add(Dense(32, input_dim=data_frame.shape[1] - 1, activation="relu"))
shapley_feature_importance.add(Dense(128, input_dim=data_frame.shape[1] - 1, activation="relu"))
shapley_feature_importance.add(Dense(64, input_dim=data_frame.shape[1] - 1, activation="relu"))
shapley_feature_importance.add(Dense(32, input_dim=data_frame.shape[1] - 1, activation="relu"))
shapley_feature_importance.add(Dense(1, activation="sigmoid"))

# Let's convert the sequential model into a functional API model
input_layer = tf.keras.Input(shape=(data_frame.shape[1] - 1,))
hidden_layer_1 = model.layers[0](input_layer)
hidden_layer_2 = model.layers[1](hidden_layer_1)
hidden_layer_3 = model.layers[2](hidden_layer_2)
hidden_layer_4 = model.layers[3](hidden_layer_3)
output_layer = model.layers[4](hidden_layer_4)
functional_model = tf.keras.Model(inputs=input_layer, outputs=output_layer)

# Let's compile and fit the model
functional_model.compile(loss="binary_crossentropy", optimizer="adam")
functional_model.fit(
    x=train_inputs,
    y=train_outputs,
    epochs=100,
    batch_size=128,
    verbose=0,
    validation_data=(test_inputs, test_outputs)
)
<keras.callbacks.History at 0x7fbfa2409720>
# Let's initialize shap explainer
shapley_explainer = shap.KernelExplainer(functional_model, train_inputs[:5000])
Using 5000 background data samples could cause slower run times. Consider using shap.sample(data, K) or shap.kmeans(data, K) to summarize the background as K samples.

# Let's calculate shap values for first 10 samples
shap_values = shapley_explainer.shap_values(test_inputs[:10])
{"ascii":false,"bar_format":null,"colour":null,"elapsed":8.313179016113281e-3,"initial":0,"n":0,"ncols":null,"nrows":null,"postfix":null,"prefix":"","rate":null,"total":10,"unit":"it","unit_divisor":1000,"unit_scale":false}
# Let's create a summary plot
shap.summary_plot(shap_values, test_inputs, plot_type="bar")

# Let's create a dependence plot for one of the shap values
for index in range(12):
    shap.dependence_plot(0, shap_values[0], test_inputs[:10])